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INTRODUCTION 



by 

Thomas Green 
Syracuse University 



The distant view of things often improves our vision. It helps 
us to see things whole and to see the parts in their relations. That 
is why history can be a great informer. It gives perspective. The 
distance of time helps us to distinguish the main events from the 
distractions I to hear the message *:hrough the static. That is also 
why there is both a challenge and a hazard wherever we attempt to 
comment on the meaning of issues in our own time. Per then we lack 
the distant view that the passing of time affords. It is hard to 
point a finger at the main event while it Is still going on. 

Right now the idea of accountability is on the rise in American 
education. Is it the main event or Is it only a side show? Is it the 
message or is it only noise? From one point of view, that is the 
question that these papers in combination help to answer. They give 
us perspective. They help to bring the parts into some whole. The 
emerging picture is^ to this reader anyway » astonlshingi startling, 
provocative and Immensely helpful. 

I do not mean that the conclusions in these papers are beyond 
dispute or that the arguments are unassailable. I mean rather that 
they Invite inquiry and provoke thought. The reader, nearly any 
thoughtful reader » is likely to find himself sometimes in agreement 
and sometimes In intense and thorough disagreement. But there Is 
documentation in these papers. The evidence accumulates and builds. 
And so the reader, if he disagrees, will find himself engaged in 
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active search of counter evidence and the task of formulating better 
and clearer arguments* 

But what l8 even more Important Is the fact that the reader will 
find himself agreeing, disagreeing and seeking clarity on matters of 
the utmost seriousness* For these papers do more than place the 
current movement of accountability and state assessment In perspective. 
They point as well as to the emergence of a different set of ^'arrange- 
ments*' for education than Americans are used to and a different way 
of framing goals for education. Instead of concentrating on a pro- 
posed solution to a current set of problems they begin to reveal 
the problems Implicit In a proposed solution. That requires a rare kind 
of perspective that not even history can provide. It looks at the 
Anerlcan educational scene not over the distance from the present 
to the past but from the present to the future. 

In the first of these papers, Miriam Clasby draws our attention 
to the Intended and latent consequences of a period of heavy ''federal 
Initiative" or "federal partnership" In education. She points out 
that the various titles of the Elementary and Secondary Education 
Act of 1965 placed a heavy responsibility on State Departments of 
Education to collect and assemble educational information on a wide 
range of topics* That same legislation also provided resources for 
strengthening the planning capabilities of State Departments* These 
matters are well known. But she also reports on what is not so well 
known, namely the emergence of increasing interstate cooperation 
of educational authorities and the rapid development of interstate 
coalitions and associations vith constituencies at the level of local 
school districts. Specifically, she draws our attention to the 
activities of the Council of Chief State School Officers, the Educa- 
tion Commission of the States, the National School Boards Associationi 
and finally, the emergence of planning bodies at the federal level 
with direct participation of such interstate agencies. 

Her documentation sketches in the outlines of a different kind 
of political structure for the formation of national educational 



policy. Despite the existence of educational systems In fifty dif- 
ferent states, there Is nonetheless, a national system of education 
In the United States, a system In which there Is scarcely more than 
ten days difference In the legal length of the school year and in 
which practice, curriculum, and pedagogy are r^aarkably uniform* 
Not so long ago It might have been argued that this ^'national system" 
of education could be described, at Itaast proximately, as consisting 
of a line of authority extending from the federal government to the 
states, and from the states to local educational authorities. But 
Dr. Clasby suggests, by only the most thinly veiled implication, 
that there is emet'glng a new system. And, in that new structure 
there is, a& it were, a third force in national policy intervening 
between state governments and federal agencies concerned with educa- 
tion. And if she is correct in this suggestion and if the trend 
continues, then the politics of education at the federal level will 
be substantially altered. It is as though our political system for 
education behaves so that any strong and sustained federal "initiative" 
can be expected to produce certain countervailing influences, perhaps 
strong lobbies and agencies having broader constituencies than any 
single State Department of Education and yet, not answerable directly 
to any state or federal educational bureaucracy. 

Such a development, if carried very far, would modify the political 
relations between the legislative and executive branches of the 
fed'sral government with respect to Issues of educational policy. 
Strong, well-staffed interstate organizations and national lay 
associations with local school district constituencies might organize 
opinion and support policies in ways that would permit direct and 
effective Congressional lobbying. It would be hazardous for any 
national administration to Ignore such a force in the political 
process of forming national educational policy. What emerges from 
this picture is a kind of coalltlonal politics In federal-state 
relations that we have not known before. If Dr. Clasby is right, 
then clearly the politics of educational policy in the future 
will have less to do with the development of appropriate and effective 
federal "initiatives*' and more to do with developing appropriate 



relations between the federal government and an array of Interstate 
bodied* 

But these papers go far beyond the exploration of this suggestive 
turn* In a second paper, Dr. Maureen Webster draws our attention to a 
particular aspect of the problcM of ^'accountability," namely, the 
growth of legislation mandating statewide assessment of schools and 
school systems. Here again, the documentation builds and eventually 
explodes into a picture of problems stemming from such legislation* 
She begins by giving us a sense of the growth of state accountability 
legislation. But she then goes on to analyze the legislation for 
statewide testing to show the array of legislated purposes and intent, 
the pattern of administration and participation and the different 
requirements for the collection and management of evaluation informa- 
tion as well as proposed uses of that information. In case after 
case, she shows that the intent of the legislation is to provide a 
more rational basis for the allocation of educational resources In 
ways that are essentially more efficient. But In a separate section, 
Dr« Webster raises serious doubts that the kinds of data mandated in 
the legislation can ever be adequate to satisfy the intended pur- 
poses of the legislation. She suggests that the legislation, depending 
on how it is implemented, contains several doubtful assumptions 
about the relative importance of what is tested, about the determinates 
of learning and about the social place of education in American 
society. She gives reason to plausibly argue that such legislation 
may, in fact, unnecessarily shape the goals of schooling itself and 
determine the shape that the policy-planning process will have to 
take. In short, in this body of ''accountability legislation" we may 
be about to give statutory definition to educational goals and to 
do so in ways that are limited to quantifiable outcomes that can 
yield data useful for legislative and administrative determination 
of efficient allocational policies. 

These are not claims that anyone can afford to react to with 
indifference. They raise vital and enormously fundamental questions 
about the very nature of education itself. In some instances. 
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state Boards of Education hava tha constitutional responsibility of 
seeing to it that "effective," "efficient," and "adequate" educational 
systems are provided for their cltiaens* What do these words mean? 
What really constitutes the moral and social grounds for the state's 
Interest in the control of education? And if that Interest extends 
only as far as to Insure some minimal level of achievement, then 
what really are the minimal skills? And do we have any reason to 
believe that these minimal skills are Included In the skills that 
state assessment legislation is Intended to monitor? 

And so there is a third paper to this set, Naomi White has 
done a provocative Job of setting down one approach to the question 
as to what could be the minimal skills to which the system of 
schooling should attend© Of course, in raising the question as to 
how we understand the minimal skills of schooling, Mrs© White Is 
raising again the question as to how the movement of "ciccovntabillty" 
is, willy-nilly, involved in defining the goals of education, and 
not simply In evaluating our success or failure In attaining those 
goals. 

Indeed, /from my own point of viewing, that is that these papers 
are about. They are concerned with setting forth some of the subtler 
aspects of how goals for the American educational system are being 
shaped and how they are being shaped under the guise of doing some-- 
thing else. The three discussions which constitute this report 
reflect a threefold emphasis guiding the work. Because the research 
has been carried out under contract to the U.S. Office of Education, 
the first chapter views decision-^making structures from a national 
perspective and suggests some implications for a federal agency. In 
several ways, these suggestions are bold ones, pointing to and laying 
the groundwork for a thorough reexamination of the role of the Office 
of Education and of federal policy in American education. But the 
federal role is constrained by the fact that constitutional authority 
and responsibility for education is reserved to the states. The 
focus on developments in state agencies, therefore, explores the 
meanings of current changes at a crucial leverage point in educational 
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decision-making. The final chapter addresses problems of the sub- 
stance of education. This emphasis reflects a value position shared 
by members of the project team which is also a fear — that current 
emphasis on equalization through finance reform and on efficiency 
in operation may cleflect attention from the fundamental Issue of 
what our children are learning. Were the suggestion not so trite, 
these papers, taken together, might be entitled Framing Educational 
Goals by Stealth: A Straightforward View of Accountability . The 
first paper documents the changing face of the federal and state 
apparatus by which goals will be hammered out, The second recounts 
the growth of the state assessment movement and its implications for 
goals, and the third paper is concerned in a more speculative way 
with what the first two might mean. Altogether it makes sense, and 
it makes serious sense. 

But in order to confront these papers vigorously, the reader, 
in my judgment, needs to examine them from a perspective that presents 
a counterargument, The details of that perspective are not clear 
to me. But it does seem that it could incorporate some of the follow- 
ing observations. 

In the first place, one should recognize that the accountability 
movement in general and the drive for state assessment in particular, 
is by no means altogether new. It can be understood as the out*- 
growth of the "testing movement" in American education going all 
the way back to World War I. At that time, the development of 
testing instrumentation converged with two other movements — the re- 
finement of techniques for business efficiency and the introduction 
of vocational education — to shape the character of elementary and 
secondary education for the rest of the century. In this context, 
as historian Marvin Lazerson has observed, testing became funda- 
mental to American education, a means of categorizing children, 
differentiating curriculum, and assuring efficiency in schools so 
that products were suitable to the industrial society. 
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But there are new developments In these movements^^the elabora- 
tion of managetnent ^'systems*' techniques, sophistication in testing 
procedures I renewed emphasis on preparation for work---that suggest 
a continuation of the tradition of the ^*cult of efficiency'* in 
American education* All of these forces » long rooted in the 
American educational experience now join together, and give 
new life and power to the factory metaphor of schools, the indus- 
trial interpretation of planning, and somewhat more indirectly, to 
the process of social stratification. 

I don't suppose that anyone really believes that this com- 
bination of metaphors can ever deal with all that we expect of 
education. In many respects, schools are not factories and educa- 
tion is not an industry. To overlook that fact would be to over- 
look the obvious. Still, it is true that what can be on the agenda 
for decision at the level of the local school, may not appear at 
all on the agenda for decision at the state or federal level. And 
though it might aeem silly to view the local school as though it 
were a factory, it is less silly to view a statewide system of 
schools as an industry. The efficient allocation of resources Just 
is a relatively larger part of the state and federal agenda for 
educational policy than it is at the local level. The industrial 
metaphor _is a relatively more plausible view to take at levels of 
high aggregation in the system. 

This leads me then to my second point. There can be no doubt 
that the data collected from state assessment will be inadequate in 
many cases to serve the statutory purposes for which it is collected. 
On that point, it seems to me that Dr. Webster is absolutely cor- 
rect. To suppose otherwise, it seems to me, would constitute eithev 
outright conceptual confusion or Just plain ignorance of how the 
American educational system works. But the relevant question may 
not be whether such data will be adequate, but whether the grounds 
for decision at the level of the State will be less ina dequate with 
such information than it would be without it. The question is not 
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posed In quitd that way In these papers* The answer might be that 
state assessment > though admittedly Inadequara as a basis for 
framing allocatlonal policies » is nonetheless an improvement. On 
the other handi the dnswe^ might be that i because of its latent 
consequences) it might not be an improvement. These papers do not 
directly address tne question in that form. Nonetheless) they con- 
tain enough documentation to strengthen the view that state as- 
sessment legislation may well not constitute an improvement in the 
long run even though it may constitute a modest gain in the rational 
basis for decisions. 



How does the demand for accountability and the emergence of 
statewide testing influence the formation of educational goals? 
That is a central theme in these papers. But it is not a theme 
pursued within the context of any general view of the "social 
logic" of educational goals. How exactly do educational goale 
Influence educational practice and under what conditions do they 
Influence educational policy? These are not questions that receive 
any detailed treatment in the documentation. They are not supposed 
to. Still) some general view of the "logic" of educational goals 
would be helpful in evaluating the claims made in these papers and 
in entering into some fruitful dialectic with these claims. There 
are three points that I want to make as a step toward such a general 
view. None of them can be fully developed) and I do not claim that 
even together they constitute all that would have to be included in 
such a view. 



In the first place) it is worth noting that general educational 
goals— as opposed to specific targets — are never formulated so that 
it is possible to tell when the goal has been attained or how far 
short we are of attainment. For example) one thing that virtually 
all educational systems must accomplish is the production of good 
citizens. But we cannot) in principle > ever know, at least in any 
short-tun period) whether that goal has been achieved. Or again) 
American schoolmen used to be fond of claiming that it is their 
goal to educate each individual to the fullest of his potential. 

O 
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But no matter how spectacular Is the achievement of students ^ we 
can never know when they are achieving at their fullest potential. 

Since It Is never possible fct us to know when we have reached 
such general goals for education i many persons have regarded such 
general goal statements as Insufficiently specific. Because they 
do not permit us to determine how far we are from their attainment » 
therefore, they are Inadequate formulations of goals for education, 
I have always found that to be a rather compelling point. It leads 
to the insistence that such general goals for education be trans^ 
lated Into rather specific targets that can be monitored and can 
be known to have been achieved or not. The Impulse to make that 
Intellectual move is bauic to the emergence of the accountability 
movement and to the development of state assessment legislation. 

However compelling that conceptual move may seem> I want now 
to suggest that it rests> nonetheless i on a conceptual mistake. 
It is derived from a misunderstanding of the nature of educational 
goals. What is peculiar about educational goals is that it is not 
their function to provide targets for attainment. Their function 
rather is to formulate consensus about what we will count as failure- 
serious failure. Consider an illustration. We cannot know when 
we have educated people to the limits of their capacities. There 
is nothing that we know of that would count as evidence that we had 
attained such a goal. Therefore i we are inclined to think that such 
a statement is vacuous. We do not know what it requires of us. It 
has no specific policy implications. We may not be able to tell 
when we have achieved the goal of educating everyone to hie fullest 
capacities. Still, there are clrcumetances in which virtually 
everyone will agree that we have failed to do so. 

The point may seem to be only a quibble. But it is not. It 
may seem that if we can agree on when we have failed to attain a 
certain educational goal, then we should be able to agree on when 
we have succeeded. But that doesn't follow at all, or rather it 



follows only If we adopt a particular, and I think erroneous, view 
of the "logic^* of educational goals. The fact is that there is 
a huge and real difference between the concern that things go very 
well indeed and the concern that they simply not go very badly • 
I au suggesting that the point at which educational goals really 
come into play is not when we have fallen short of the best, but 
when we have fallen short of what is the least that is tolerable. 
Under these conditions, the general goal that we should educate 
each person to his fullest capacities is no longer vacuous. On 
the contrary, It means rather specific things and JLt has direct 
and pointed Implications for policy and for the allocation of 
resources. It means that the curriculum should not be too narrow, 
that facilities should not be too Impoverished and that standards 
should not be too low. The specific meaning of phrases like "too 
narrow," "too impoverished," and "too low" will, of course, be 
defined differently in different settings. But it will not be 
defined as something merely short of vhat is good, but as something 
short of what is acceptable. Whenever such a goal is invoked 
in a community, it will be because rather specific things are re- 
garded not as less than the best, but as less than what is tolerable. 
People will be saying either that the curriculum is too narrow 
(add a French class) or that the facilities are too inadequate (add 
a homeroom) or that the children are not being pressed to do what 
they are capable of (don^t underestimate our kidsl). The general 
goal then gets translated into specific targets naturally, quickly, 
and with clarity; and that translation is precisely what is re- 
quested in the demand for "accountability." My point is that the 
first step in understanding the "social logic" of educational goals 
is to recognize that it is not their function to tell us What in 
any detail we are to count as the best state of affairs. It is 
their function rather to tell us what we are to count as relevant 
in determining when things are intolerably bad. And this they 
do tolerably well. 
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If these observations are anywhere near being ^'on target," 
then there are certain things that flow Immediately to provide still 
another perspective on the papers tn this report, still another 
kind of distance* In the first place. If I am anywhere near right 
in this view of educational goals, It follows that goals In educa- 
tion are a different sort of thing frow goals In Industry* And 
that Is a rather specific respect in which the factory and Indus- 
trial metaphors do not apply to education* In business, in 
engineering, and perhaps even In government, the declension for the 
word ••goal" would be ''good," ''better,'' and "best"} but In education 
It might well be "not good," "worse," and "absolutely Intolerable*" 
There Is an Irony to this* For the conception of educational goals 
that Is adopted In these papers Is not the one that I have advanced* 
On the contrary, the conception of the "logic" of educational goals 
found In these papers Is the conception appropriate to Industry. 
The authors are quite consistent In holding to that view, and for 
good reason. That Is, distressingly enough, the view embodied In 
the purposes and Intent of the legislation^ 

I believe that Dr. Webster Is correct In suggesting that the 
data gathered under statewide testing legislation will be Inadequate, 
on the whole, to serve the purposes and Intent embodied In the 
statutes. But one must confess that no amount of Information Is 
likely to be sufficient for that. Still, she has something rather 
more serious In mind. If statewide testing legislation results In 
attempts to define educational goals along lines of "good, better, 
and best," then that will Indeed be something new. But I suspect 
that It will not work. Sooner or later. In the process^ of Imple- 
mentation and with the hindsight provided by some experience, we 
shall have to face up to the peculiarities in the logic of educa- 
tional goals. The data will be used not to identify what Is better 
and best, but what is bad, worse, and intolerable. It will be 
used to give us quicker and more satisfactory notice of where we 
are failing, and failing badly. That is a gain, but it is not 
quite what the proponents of "accountability" think they are pro- 
Q motlng. 
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Secondly » I have argued that there Is an itaportant and real 
difference between the concern that things go very well indeed » 
and the concern that they simply not go very badly. The difference 
can be construed as the difference between the educational goals 
that a concerned parent has for his children and the goals that 
the state has for its citizens • Parents typically will seek what 
they think is desirable for their children even when they do not 
believe it is necessary. But the state will typically seek what 
is desirable in education only If it ^an also be shown to be either 
politically or logically necessary. Parents will seek to assure 
their children of the best they can. The state will seek to guarantee 
only what is minimal* That is precisely why the third of these 
papers is absolutely crucial* There is indeed a question as to 
whether the skills to be assessed under statewide testing legisla- 
tion are Indeed the skills appropriate to the staters legitimate 
interests In education. Mrs* White shows us that schools and 
school systems might be held accountable for minimal skills that 
are not typically mandated by existing legislation) and in doing 
SO) she also raises hard questions about how we are to understand 
the definition of minimal skills and how we are to understand the 
limits of the state's legitimate Interests. 



As a possible counterpoint, I would like to suggest briefly yet 
another way in which this problai might be approached. Nothing can 
be a goal in any precinct of human affairs if it Is already attained. 
That may seem obvious, but the obvious is often worth noting. In 
this case, the obvious helps us to understand why it is that in the 
'*social logic** of educational goals — as opposed to the **pdi^ental 
logic" — something can become a goal when its absence becomes trans- 
parently clear. It also helps us to see why it is that educational 
goals typically deal not with the specification of what is beet or 
even pood, but with what we are to count as relevant in the judgment 
that things have gotten Intolerably b^d. Now, of course, what is 
regarded as intolerable may change. If we have a society in which 
everyone learns to read, then the fact that few enjoy it might be 

ERLC 



regarded as Intolerable. In that case the schools might be saddled 
with the responsibility of teaching reading bo that It Is enjoyed > 
and that tnlght be regarded as among the minimal responsibilities 
of the schools* But If we have a society In which substantial 
numbers of people do not even learn to read in the literal sense 
of basic functional literacy > then I suspect that the fact that 
few enjoy reading will be regarded as tolerable and the minimal 
responsibilities of schools will not include such a goal. And if 
there are circumstances in which children in school have neither 
security of person nor property, then I su&pect that even the 
teaching of basic literacy might give way to certain other goals. 
But at that point » the question might be raised as to whether the 
goals of securing safety of person and property are in fact educa- 
tional goals. 

The point I am driving at is that, given the perspective X have 
sketched on the "logic*' of educational goals, we have also a way of 
defining the minimal skills that schools can legitimately be held 
accountable for. The class of minimal skills will be defined as that 
set of skills such that the failure of the schools to transmit them 
would constitute sufficient grounds for saying that the schools are 
intolerably bad. They would then become educational goals. Such a 
definition, of course, defines a highly flexible set of skills. 
What specific skills actually fall within that set will vary from 
community to community, from society to society, and from time to 
time. It will also have much to do with what people believe they 
can take for granted that the school will successfully accomplish. 

In our own society, I suspect that the 3R*s plus a certain 
kind of citizenship education fall within the set of minimal skills. 
They are also the skills that, on the whole, are included in areas 
to be assessed under statewide testing legislation. But it seems 
to me also that there is an Interesting thing happening to the level 
of what is regarded as intolerable In America. There is increasing 
evidence that with respect to these basic areas of education, a 
growing part of the population assumes that the schools can be 
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successful. They tend to regard those areas of work as the functions 
of the schools and not the goals of educate on. The concern is 
growing that however well the schools may do In discharging these 
basic educational functions i nonetheless! the schools ate Intolerably 
bad If children are boredi unimaginative* and unfulfilled. In 
short I the quality of life In schools Is likely to emerge as an 
educational {joal* as something for which schools may be legitimately 
held accountable. And yet It Is something that receives little 
attention In the state assessment legislation ! To say that Is to 
arrive at yet another central theme of these papers from yet another 
direction and to suggest the range of Issues that the discussion 
Invites each reader to confront. 
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Chapter One 



ACTION AND ACTORS ON THE NATIONAL SCENE 



by 

Miriam Clasby 



PART ONE 
INTRODUCTION 



A New Federal Role 

A number of Influences have focused national attention on 

education* The rapid expansion of the school-age population in the 

i950's and 1960*s, coupled with i>oarlng costs > created financial 

pressures touching every segment of the population. This period, 

however, also marked a dramatic shift in the role of the federal 

government in elementary and secondary education — a shift that has 

not been given the careful study It deserves « Analyses that have 

been undertaken tend to focus on discrete components of federal 

action in separate titles of the Elementary and Secondary Education 

2 

Act (ESEA) or isolated issues such as finance or politics. One 
of the most ambitious tasks undertaken to date is the investigation 
of the impact of federal aid to education on the finances of ele- 
mentary and secondary public schools conducted under the direction 

3 

of Michael Kirst and Joel Berke. This research traces the flow 
of major federal programs in five hundred seventy-five school dis- 
tricts In six states over a four-year period. The large picture 
of federal activity in education is, however, only beginning to 
emerge. 

In a provocative study published In 1963, Roald F. Campbell 
and Robert A, Bunnell examined "nationalizing Influences on secondary 
education" by focusing on the effects of four national programs on 
different types of high schools. They concluded that four programs 
(National Science Foundation, College Entrance Examination Board, 
National Defense Education Act, and National Merit Scholarship) 
impacted in different ways on different kinds of schools, and that, 



In general, such programs were having a decided Influence on secondary 
education. The authors observed: 

The debate Is now taking place relative to the pos- 
sibility of large scale support by the federal govern- 
ment for total elementary and secondary public school 
programs. This represents another^ thouf^h untried , 
type of national Influence • If we can learn something 
from the typea of Influence already In existence, we 
may be In a better position to evaluate the proposed 
federal programs and thus to participate Intelligently 
In the decision, (p. 10) 

Campbell and Bunnell were attempting to anticipate the consequences 
of massive federal legislation (and It is unclear to what extent 
their efforts were formative in shaping the direction of that action)* 
Given the fact that federal support for elementary and secondary 
education has been operative since 1965, the Intent here is to outline 
some aspects of this hitherto "untried type of national influence," 

The administration of ESEA has set in place a variety of new 
management procedures. Some are required by legislation; others are 
established in administrative guidelines. This discussion of the new 
federal role in education is sharply limited. It focuses only on new 
organizational and administrative activities insofar as they impact 
on state agencies. It examines ways in which this federal activity 
has contributed to the climate of educational dec is ion -making, and 
identifies variations in state responses to this influence. The 
analysis does not address issues of the substance of federal programs 
or areas of priority. The purpose is, rather, to surface some of 
the indirect and unintended influences of federal activity on state 
operations, to point to relationships rather than to causation, and 
to suggest dimensions of federal activity which have served as a pre- 
condition to current state activity. 

Part Two examines administrative requirements for the Elementary 
and Secondary Education Act and related State Education Agency (SEA) 
activity. Under the rubric "Scope of Activity," special attention 
^ will focus on data--gatherlng and planning activities required by 
E[^C federal legislation and subsequent changes in the structure and 



functions of SEA's, A discussion of ''Modes of Participation" will 
describe legislative and administrative requirements for advisory 
groups and related activity at the state level, including both iatra- 
state and interstate patterns of participation* 

Part Three is an exploratory piece charting the changing 
functions of three national groups^ the Council of Chief State School 
Officers > the Education Commission of the States, and the National 
School Boards Association » These changes are examined in the light 
of federal activity which has both precipitated opposition and sup- 
ported activities in these groups which have the capability of impact- 
ing on education at the federal, state, and local levels* 

Part Four draws on the previous discussions to suggest three 
needs which are implied for federal policy-makers: 

a) to identify changes in state agencies which are concurrent 
with shifts in funding patterns and to anticipate the consequences of 
these changes for future federal policy; 

b) to identify the rationale for current relationships with 
national interest groups and to explore alternative policies; 

c) to review and synthesize federal experience with strategies 
for change in elementary and secondary education and to translate this 
into policy options for federal action and into policy analysis for 
stQte use. 
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PART TWO 

FEDERAL ADMINISTRATIVE REQUIREMENTS AND SEA ACTIVITY 



The Elementary and Secondary Education Act of 1965 (P.L, 
89-10) represents the major federal thrust in elementary and 
secondary education during the past decade. Therefore > it serves 
as a touchstone for identifying major features of federal activ- 
ity. The magnitude of the funding levels and the range of admin- 
istrative responsibilities have Impacted on the roles of SEA's 
in ways that have not yet been adequately examined.^ The com- 
plexity of the individual ESEA titles has resulted in multiple 
patterns of administration — state control, state sign-off, state 
by-pass — that severely complicate the problem of analyzing in- 
fluences on administrative patterns at the state level. Some 
indicators, however, do suggest broad influences on state opera- 
tions related to a) the scope of administrative activities, and 
b) patterns of participation within states and between states. 

Scope of Activity 

Recent efforts to Improve educational decision-making have 
emphasized the need for an adequate Information base, procedures 
for planning and evaluation, and processes for long-range com- 
prehensive planning. Each of these needs will be examined in 
relation to the impact of ESEA requirements on state activity. 

A . Federal ReguitementS! Data-gathering and Planning 

Data-gathering . The federal education agency was originally 
established for the purpose of data collection and this has re- 
mained its most permanent function for more than a century. The 



passage of ESEA, however, Imposed significant additional data- 
gathering responsibilities on the Office of Education and these, in 
turn, generated new demands on state agencies, (After several years 
of effort, the official reporting forms which states must use each 
year have been reduced to a package weighing approximately ten 
pounds i) 

Each title of ESEA requires its own information base and 
reporting pattern, imposing on the states a monumental task of data 
coordination. In addition, because Title I was targeted to meet the 
educational needs of low- income families, the legislation specifies 
precise criteria for the allocation of funds to local education 
agencies. Three major groups in the 5-17 age category are identi- 
fied: children of families having an annual income less than the 
low-income factor, those In families receiving aid under Title IV 
of the Soctal Security Act, and those in institutions for neglected 
or delinquent children or being supported in foster homes with, pub- 
lic funds (Section 103). Other provisions specify Indian childreni 
those residing in the school district of another agency, the handi- 
capped, and children of migratory agricultural workers. The legis- 
lation requires that calculations be made on the basis of most 
recent satisfactory data available from the Department of Commerce, 
with special estimates of the number of children in each group. 

More recent federal efforts to insure comparability of services 
in Title I districts require still finer level of detail. State 
educational agencies have been instructed to require each local 
agency to provide data on each school served by Title I and all 
other schools showings 1) the average daily membership; 2) the average 
number of assigned certified classroom teachers; 3) the average 
number of assigned certified instructional staff other than teachers; 
4) the average number of assigned noncertified staff; 5) the amount 
expended for instructional salaries; 6) the amount of such expenses 
for longevity pay; and 7) the amount expended for other instructional 
costs, such as the costs of textbooks, library resources, and other 
instructional materials. In addition, data must be submitted giving 
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the per-pupil averages fot each of these Items based on average 
dally memberships ,6 

The original Title I legislation also requires that annual 
reports to the state education agency include evidence that 
"effective procedures, Including provision for appropriate objec- 
tive measures of educational achievement, will be adopted for 
evaluating at least annually the effectiveness of the programs In 
meeting the special educational needfi of educationally deprived 
children, (Section 105) 

States, there fore, have been rcsponsibte for approving applioa-- 
Hons requiring massive information gathering and reporting pro- 
cedures focused on detailed demographic data and including evidence 
of outcomes in terms of stvAent performance. 

Planning and Evaluation . Conceptual clarity may exist among 
experts on the roles and techniques of planning and evaluation, but 
there is a muddiness at the operational level that militates against 
broad generalizations. At the state level, for example, evaluation 
may or may not be related to a systematic planning process; state- 
wide testing may thus function either as a simple performance 
measurement or as a component of a needs assessment feeding Into 
a planning process. On the oLher hand, the planning operation It- 
self may function as an Independent unit directly concerned with a 
specific phase of operation such as resource allocation or educa- 
tional program development. Alternatively, the planning may be 
comprehensively designed to integrate and coordinate all phases of 
operation. 

ESEA has required and supported planning and evaluation activ- 
ity at multiple levels. The Title I requirement for objective 
measures of achievement supports the notion of testing as a simple 
performance measure. Although the legislation allows other measures 
of effectiveness, only this one is specified. Title III> on the 
other hand, requires states to formulate state plans for administration 
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and to demonstrate that the proposed programs respond to state edu 
catlonal needs: 



Section 305 (b). The Commissioner shall approve a 
state plan t or modification thereof, if he determines 
that the plan submitted for that fiscal year — 

(1) sets forth a program ( including educational 
needsj and their basis> and the manner in which the 
funds paid to the state under this title shall he used 
in meeting such educational needs ) under which funds 
paid to the state under Section 307 (a) will be ex- 
pended solely for the improvement of education In the 
state through grants to local educational agencies for 
programs or projects in accordance with Sections 303 
and 304. (Emphasis added] 

In this case, the legislation has supported both planning and 
related types of needs assessment* 



The Amendments of 1967 (P.L, 90-247) introduce new authoriza- 
tion for Planning and Evaluation in Title IV: 

Section 402. There are authorized to be appropriated, 
for each fiscal year for which appropriations are 
otherwise authorized under any title or act referred to 
in Section 401, such sums as may be necessary to be 
available to the Secretary, in accordance with regula- 
tions prescribed by him, for expenses, including grants, 
contracts, or other payments, for (I) planning for the 
succeeding year programs or projects authorized under 
such Title or Act, and 2) evaluation of programs or pro- 
jects so authorized. [Emphasis addedj 

In 1970, each state received $96,000 for planning and evaluation^ 



Comprehensive Planning ^ Title IV, Section 402 of the Amend- 
ments of 1967 which provides added incentive for comprehensive 
planning at the state level, highlights a dilemma inherent in the 
current federal role. Federal programs themselves are fragmented 
and unrelated^ imposing enormous burdens of coordination on state 
and local levels. Furthermore, federal authority can only be 
attached to federal programs; the federal government cannot requir 
comprehensive planning at the state level. 



The original 1965 legislation places a priority on comprehensive 
planning at the state level. In identifying the types of projects 
appropriate for funding under Title V, the ESEA legislation encourages 
comprehensive planning by pointing first to: 

educational planning on a statewide basis, including the 
identification of educational problems, issues, and needs 
in the state and the evaluation on a periodic or continuing 
basis of education programs in the state. (Section 503) 

The choice and design of such projects is left to the states, subject 
to federal approval.^ 

Whatever administrative priorities my have been supported at 
the congressional or agency level, the distinctions in federal action 
in the three areas are clear: 1) firm and fixed requirements for 
informtion gathering and reporting including detailed demographic 
data and obdective measures of Title I student perfoimw^^ce; 2) general 
requirements for state plans in Title Ills funding and opportunities 
for state planning and evaluation of federal programs through 196? 
Title IV amendments; and 3) support and encouragement for compre- 
hensive planning in Title V authorizations , but no enforceable 
requirements, 

B, SEA Activity: Data-gathering and Planning 

Changes in state agencies stimulated by ESEA legislation are 
evident in 1) personnel build-up, 2) new functions, and 3) new struc- 
tures. 

Because of discrepancies in reporting systems, the precise 

figures on personnel increase in state agencies are not firm. Official 

figures indicate an increase from 14,720 employees in state agencies 

in 1965 to 21,697 employees in 1970, although USOE staff report that 

9 

state staffing doubled during this period* 

Changes in functions are suggested by reports made by states in 
a survey of State Educational Assessment Programs undertaken by 
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Educational Testing Service in 1971,^ Table hi (pp, 199-201) Iden- 
tifies twenty-five states which reported expanded statewide assess- 
ment programs then operational or projected. 

The assessment programs described by the states range from 
simple student testing to comprehensive five-year projects. Twenty- 
five states reported testing or assessment programs which were sig- 
nificantly broader than the requirements of Title I or Title III. 
Almost all these programs (twenty-two) were initiated with federal 
funds. Programs in twelve states were funded solely with ESEA 
monies; programs in ten states used ESEA funds In combination with 
state and/or local funds. Six states specifically noted that these 
expanded programs originated from Title I or Title III requirements. 
Updated ETS information indicates that more than 75% of current state 
assessment programs rely totally or partially on federal funds. 

These functional changes, moreover, are integrated with organi- 
zational changes. In a 1971 survey undertaken by the Missouri State 
Department of Education, thirty-four states reported on planning 
offices in state education agencies.'''^ Twenty states (59%) indicated 
that planning units were established in 1970, the year in which Title 
IV, Section 402 funds became available for such purposes. A summary 
of SEA expenditures for comprehensive planning and evaluanion (Table 1.2) 
suggests that states have attempted to focus and coordinate other 
state and federal funds beyond the $96,000 allocated under Titl'^ IV 
to each state. Figures indicate a median of $156,000 expended on 
state planning and evaluation activities. 

Evidence is mounting that federal activities have introduced 
n&i) personnel y supported new activities^ and facilitated organiza-- 
tional change in state education agencies. 
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TABLE r2 



TOTAL FUNOS fOH COWPREHENSiVE RANNINQ AND EVALUATION 
(SECTION 402. PUBLIC LAW 90-2471. BV SOURCE. FIRST GRANT PERIOD; FISCAL YEAR 1970' 



Stat* or Oth*f Anal 


TitttIV 


Perc«f>t 


Othtf Ftd. 


^rMnt 


Stata 


^rC4nt 


Total 


»«C.402 




(jndi 




fur>di 




Grand Total 


15,000,000 


35,0 


$6,645,286 


47,8 


$2,456,134 


17.2 


$14,301,419 




de.ooo 


66. t 


74,336 


43.4 


900 


0,6 


171,236 




96.000 


46.2 


112,000 


53.8 




0.0 


208,000 


Arliona 


96.000 


100.0 


, 


0.0 




0.0 


96,000 




96.000 


67.1 


61^ 


30.5 


20.640 


12.4 


168.100 


CalifOi'pla . . < • 


96,000 


82,8 


20.000 


17.2 




0.0 


116.00C 


Colofado 


96.000 


322 


125.038 


41.9 


' 77!454 


25.9 


298,492 




96,000 


74,3 


33.214 


25,7 




0.0 


129,214 




96.000 


42.8 


112.000 


49.9 




7.3 


224,250 


Oiltrict of Cotjmblik »• 


67.000 


77.9 


19.069 


22.1 




0.0 


86.059 




96.000 


36.3 


154.605 


58.6 


13700 


6.2 


264.305 


GaofQii 


96,000 


26.7 


93.551 


26.1 


169.541 


47!2 


369.092 




96.000 


62.9 


70.000 


38.6 


16,500 


8.5 


181,500 




96.000 


73,6 


25.683 


19.7 


8,667 


6.7 


130,350 




S6,000 


60,2 


36.100 


22,6 


27.500 


^7.2 


159.600 


lodiaoa . . . . 


96,000 


70,1 


25.000 


18.2 


16.000 


11,7 


137.000 




96,000 


68.9 


34.000 


20.9 


33.000 


20,2 


163.000 




96.000 


61.6 


60.000 


38.6 





0,0 


156,000 




96,000 


73.6 


4.024 


3.1 


30,000 


23.1 


130,024 




96.000 


63.2 


32.000 


21.0 


24,000 


15.8 


152.000 




96,000 


66.0 


65.496 


37.5 


13.C65 


7.6 


174.561 




96,000 


38.6 


110.000 


44.2 


42.936 


17.2 


248.936 




96.000 


68,1 


45.000 


31.9 




0.0 


141.000 


MichlQan 


96.000 


76,5 


4,908 


3 9 


24,519 


19.6 


125,427 




96.000 


49,6 


83.680 


431 


14,320 


7.4 


194,000 




96,000 


563 


61.515 


36,1 


13.000 


7.6 


170.515 




96,000 


91.4 


6.000 


5,7 


3.000 


2.9 


106.000 




96.000 


67.6 


46,266 


32,5 




0.0 


142.265 




96.000 


63.3 


75.300 


41,8 


8.700 


4.9 


180.000 




96.000 


62.7 


86.000 


47^ 




0.0 


182,000 


Nftw Hampthirt 


96.000 


41.7 


93.500 


40,6 


40.800 


17.7 


230,300 




96,000 


69.3 


35.000 


21.6 


j 1.000 


19.0 


162,000 




96.000 


58,9 


57.425 


35.3 


9,500 


6.8 


162.926 


Vof k 


96.000 


2,2 


3,770,000 


85.8 


625,000 


12.0 


4,391,000 




96.000 


39,0 


146.000 


59.4 


4.000 


1.6 


246,000 




96.000 


64,6 


36.500 


24.6 


i6.ogo 


10.8 


148,500 


Ohio 


96.000 


49.0 


100.000 


51.0 




0.0 


196,000 


Oklahofna . ■ » ■ » 


96.000 


74.7 


19.600 


15,2 


13,000 


10.1 


128.600 




96.000 


35.0 


63.200 


23.1 


114,700 


41.9 


273)900 




96,000 


70.6 




0,0 


40,000 


29.4 


136.000 




96,000 


48.6 


* ?o!ooc> 


35,3 


32,000 


16.2 


198.000 




96.000 


10.1 


223.673 


23.7 


625.171 


66,2 


944.844 




OA AAA 






J T.< 


'Ui AAA 


16.4 


183,300 




96,000 


59.3 


46.000 


28.4 


20.000 


12.3 


162.000 




96,000 


71.2 


22.791 


16,9 


16.040 


11,9 


134,831 




96.000 


63.4 


55.500 


36.6 




0.0 


151.500 




96.000 


759 


25.500 


20.2 


5,000 


3.9 


126.500 




96.000 


66.0 


2M12 


15.2 


27432 


18.6 


145,544 




96.000 


24J 


61.037 


15.4 


238.246 


60.3 


395.283 




96.000 


42.9 


105,208 


47.0 


22.700 


10.1 


223.908 




96,000 


G6.& 


34.443 


23.8 


14,000 


9.7 


144.443 




96,000 


58.0 


42.350 


26.6 


27.092 


16.4 


165.442 


Amorkao Samoa . . 


16.S00 


54.6 




0,0 


13,600 


45.2 


30,100 




16.600 


27.3 


44.000 


72,7 




0.0 


60,500 




67.000 


60.6 


29,112 


26J 


14.471 


13.1 


110.583 


Virgin Islandi 


16.500 


100.0 




0.0 




0.0 


16.500 




16.500 


42.3 


19.000 


48.7 


3.500 


9.0 


39,000 




96.000 


34.0 


56.500 


48.8 


16.000 


17.2 
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* Source; State Oapaffn^ntt of Educatton and Federal Prograrr^i; Annual Raport f ileal Year 1970. OHEW f^ublicatlon 
No. {OE) 72-68. (TabTa 16. p. 971 



Patterns of Participation 



A. Federal Requirements; Advisory Councils 

Furthermore! various mechanisms for participation have been 
specified In federal programs, some required by legislation, others 
established by administrative guidelines. 



Federal Level , The simplest mode Is the National Advisory 
Council format which was mandated for Title I: 

Section 134. (a) The President shall, within ninety 
days after the enactment of this Title, appoint a 
National Advisory Council on the Education of Disad- 
vantaged Children for the purpose of reviewing the 
administration and operation of this Title > Including 
its effectiveness In Improving the educational attain- 
ment of educationally deprived children, and making 
recommendations for the Im provemen t of this Title and 
Its administration and operation * These recommendations 
shall take into consideration experience gained under 
this and other federal educational programs for dis- 
advantaged children and, to the extent appropriate, 
experience gained under other public and private educa- 
tional programs for disadvantaged children. 

(c) The Council shall make an annual report of Its 
findings and recommendations (Including recommendations 
for changes In the provisions of this Title) to the 
President and the Congress not later than January 31 
of each calendar year beginning after the enactment of 
this Title. The President Is requested to transmit to 
the Congress such comments and recommendations as he may 
have with respect to such report. [Emphasis added] 

Similar provisions were made for Titles III, V and VII. In 
general, National Advisory Councils are charged with the responsi- 
bility of reviewing the administration and operation of the program, 
making recommendations for Improvement, and submitting these recom- 
mendations to the President and Congress. There Is little specifi- 
cation In Titles I and III concerning membership. Title III suggests 
the Inclusion of persons familiar with the educational needs of the 
nation, with the administration of state and local educational pro- 
^ grams, and those representative of the general public. Title VII 
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specifies that at least four bf the nine members be educators expert*- 
enced in dealing with the educational problems of children whose 
native tongue is a language other than English* 

State Level * Title III legislation specifies the establishment 
of State Advisory Councils following the same patteiti as for the 
National Advisory Councils, The statute indicates that membership 
should be representative of elementary and secondary schools, insti- 
tutions of higher education and areas of professional competence in 
special education. The functions of the council are to advise the 
state on policy matters, to review and make recommendations on applica- 
tions, to evaluate programs and to submit a report of activities, 
recommendations and evaluations to the Commissioner of Education and 
the National Advisory Council* (P,L, 89-10, Section 305) 

Local Level . Through administrative guidelines, Title I has 
also set forth requirements for the establishment of advisory councils 
in each school district receiving Title I funds. The mode of the par- 
ticipation has become more sharply defined over time. Authority for 
these regulations rested on the provision of the original legislation 
that state educational agencies approve grants to local agencies only 
after determining that they were "consistent with such basic criteria 
as the Commissioner may establish." (Section 205 [a]) The revised 
criteria which became effective on July 1, 1967 specified the involve- 
ment of parents at the local level; 

The Title I program includes appropriate activities or 
services in which parents will be involved . The applicant 
should demonstrate that adequate provision has been made 
in the Title I program for the participation of and 
special services for the parents of children involved in 
the programs. The employment of parents in the Title I 
projects is but one way to implement this provision. 
The primary goal of such activities and services should 
be to build the capabilities of the parents to work with 
the school in a way which supports their children's well- 
being, growth, and development,^ (Emphasis added] 

The guidelines of July 1968 added further explanation of this 
criterion: 



The criteria also require, as indicated in item 3.1 
and the discussion following the Item, that the same 
groups, agencies, parents and others be involved in a 
comprehensive analysis of the resources available to 
meet those needs and in the development of a compre- 
hensive compensatory educational program for the co- 
ordinated use of Title I funds and of the resources 
from other programs and agencies. 

To carry out effectively the intent of these criteria, 
each Title I applicant must have an appropriate organi- 
zational arrangement. This means, in effect, that local 
advisory committees will need to be established for the 
planning, operation, and appraisal of a comprehensive 
compensatory educational program . 13 (Emphasis added) 

Suggestions were made for the composition of the local advisory 
committee; 

It is suggested that at least 50% of the membership of 
the committee consist of parents of disadvantaged child- 
ren attending schools serving the area where projects 
will be conducted, representatives of the poor from the 
Community Action Agency and parent members of Head Start 
advisory committee, if there is a Head Start project In 
the community, and representatives of other neighborhood- 
based organizations which have a particular interest in 
the compensatory educational program. ^3 (Emphasis added] 

The principal functions of the committee were also specified: 

a. Supply information concerning the views of parents and 
children about unmet educational needs in the Title I 
project areas and establish priorities among these needs. 

b. Recommend a general plan for the concentration of funds 
in specific schools and grade levels. 

c. Participate in the development of proposals which are 
particularly adapted to bridging the gap between the needs 
of the pupils and the curriculum of the school. 

d. Make written concurring or dissenting comments to be for- 
warded with the application. 

e. Act as a hearing committee for suggestions to Improve the 
compensatory educational program. 

f . Hear complaints about the program and make recommendations 
for its improvement. 

g. Participate in appraisals of the program. 
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Following a period of Intense and conflicting pressures from 
both education lobbies and community groups, on April 21, 1971 addi- 
tional requirements were proposed which related to public Information 
and parental Involvement at the district level. The legal form of 
these regulations, reported on October 14, 1971| sets out clear 
guidelines for dissemination of information: 

Each application by a local educational agency for a 
grant under Title I of the Act shall include specific 
plans for disseminating information concerning the 
provisions of Title I, and the applicant's past and 
present Title I programs, including evaluations of such 
programs, to parents and to the general public and for 
making available to them upon request the full text of 
current and past Title I applications, all pertinent 
documents related to those applications, evaluations 
of the applicant's past Title I projects, all reports 
required by 116.23 to be submitted to the state educa- 
tion agency, and such other documents as may be reason- 
ably necessary to meet the needs of such parents or 
other members of the public for information related to 
the comprehensive planning, operation, and evaluation 
of the Title I program but not including information 
relating to the performance of identified children and 
teachers I Such plans shall include provisions for the 
reproduction, upon request, of such documents free of 
charge or at reasonable cost not to exceed the addi- 
tional costs incurred which are not covered by Title I 
funds or provisions whereby persons requesting such 
copies will be given adequate opportunity to arrange 
for the reproduction of such documents. Federal Register > 
Vol. 36, No. 81 (April 27, 1971), Section 116.17. 
[Emphasis added] 

The requirements for [parental involvement specified that each local 
agency 

(1) shall describe how parents of the children to be 
served were consulted and involved in the planning 
of the project , and 

(2) shall set forth specific plans for the continuing 
involvement of such parents in the future planning 
and in the development and operation of the project. 

[Emphasis added} 

In addition, regulations for the establishment of the advisory 
council outlined additional criteria including: 

(1) measures to Insure representatives in the selection 
process 



(2) assurance of complete and free Information to the 
council 

(3) Involvement of the council in future planning 

(4) adequate opportunity to consider the Information 
and make recommendations 

(5) opportunity to review prior evaluations 

(6) specific provisions for informing and consulting 
with parents 

(7) adequate procedures to respond to complaints and 
suggestions from parents and parent council 

(8) opportunity of all parents to present views. 

Introductory comments to these regulations to clarify their intent 
summarized some of the debate which had been generated by the Initial 
publication and indicated the federal stances 

The regulation [on parental involvement] is designed to 
give each local educational agency sufficient flexibility 
to establish a parent council that is appropriate for 
its school district and to in&jre that the council has 
the information and opportunities it needs to be effective » 
(Section 116) 

Federal authority to establish this kind of guideline was re- 
inforced by the Amendments of 1967 which gave broad authority to 
the Commissioner of Education to require such participation at his 
discretion. States^ ther^e fore ^ have been required both to interact 
with state-level oohkoHb and to oversee the organization of advisory 
aounoile at the local level. 

B. SEA Activity; .Patterns of Participation 
1. Intrastate Activities 

There is a dearth of analysis on modes of participation in state 
agencies and only a tenuous link here with federal activities. Still, 
there are indications that various new mechanisms for participation 
are being introduced at state levels. These mechanisms provide for 
both state-level participation and for local-level involvement • 
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State Level . The Interview guide for the ETS study of State 
Educational Assessment Programs did not directly raise the question 
of participation^ but in discussing the "Planning of the Program," 
the schedule askss 

Who determines how the program is conducted and what 
changes will be made In the nature of the program 
(committee drawn from state universities, governor's 
office, teachers* association, independent organiza- 
tions)? How is the planning done? 

Responses to this question provide evidence on participation 
that tends to be sparse and ambiguous, but Table 1,1 (pp. 199-201) sug- 
gests that fourteen of the twenty-five states in this group utilize 
some mechanism for participation. Four general patterns are reported: 

- Regional and Local Educational Personnel 
(Arkansas, South Carolina, Texas and Utah) 

- Title III Needs Assessment Groups 
(Arkansas, Kentucky, Louisiana and New Jersey) 

- Official State Groups 

Delaware — Educational Accountability Council 
Kansas— State School' Practices Efficiency Committee 
North Carolina — Proposed Education Development Council (citizens) 
Pennsylvania — Board Committee on Quality Education an^ Statewide 
Advisory Committee 

- Special Committees 

Colorado — Cross-sectional advisory group of legislators, 

citizens and professional educators for pilot planning 
Florida — Various groups for program review 

Utah — Groups of citizens, educators and representatives or state 
organizations 

Vermont — Local participation of parents and citizens 

Since these expanded assessment programs represent state program admin^ 
istration rather than administration of a single discrete federal pro- 
gram^ the patterns of participation suggest that states aX'e initiating 
a variety of participatory mechanisms for state decision-making. 



State-Local * The summary of state legislation for accountability 
prepared by the Cooperative Accountability Project provides several 
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examples of modes of local participation required by state legisla*- 
tionj^^ 

- Colorado Legislation^ 1971, establishes both state and 
local groups specifying in detail both term of office 
and membership of the state committee* Local districts 
are required to appoint an accountability committee con- 
sisting of at least one parent, one teacher, one school 
administrator, and a taxpayer from the district* 
(Educational Accountability Act of 1971^ Article Al) 

- Virginia Legislation, 1971, directs that "the principal 
shall involve the community and his staff in the prepara- 
tion and implementation of an annual school plan which 
shall be consistent with the division-wide plan and which 
shall be approved by the division superintendent/' (//845) 

- New Mexico Legislation, 1971, directs the State Board of 
Education to recognize Educational Evaluation Committees 
in various school districts without current evaluation 
procedures and indicates that each committee shall "con- 
sist of ten members, no more than two of whom shall be 
teachers within the district, and the remainder of which 
shall be parents or concerned citizens in the school dis- 
trict, but having no position of responsibility within 
the public school system in the district*" Duties of the 
Committee include: 1) visiting schools in the community 
during their usual operation, 2) observing the physical 
plans and educational facilities in the school district, 

3) conducting open discussion at public raeetings on all 
issues relevant to the schools for each grade level, 

4) etc, (Senate Memorial 40, January, 1971) 

- Illinois, 1972, requires a system of financial planning, 
management and control which includes "the establishment, 
with maximum community, school board, staff and admin- 
istration participation, of measurable goals and objectives 
for education within the district. (Senate Bill No, 1548) 

- Although Connecticut has no statewide planning mechanism, 
legislation on the development of Innovative Educational 
Programs, 1971, requires: "provision for direct participa- 
tion by members of the communities and students to be 
served by such experimental educational projects, in plan- 
ning, policy-making and service functions affecting such 
projects," (Public Act No, 430, June 6, 1971) 

Efforts to encourage participation have also generated a variety of 
materials and programs in states. In California, the Joint Committee 
on Educational Goals and Evaluation in the state legislation, under 
Assemblyman John Vasconcellos , has produced two volumes to serve as 
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guidelines for local participation under the general heading Education 
for tjie People . Volume One presents ''Guidelines for Total Community 
Participation in Forming and Strengthening the Future of Public Ele- 
mentary and Secondary Education in California/' and the companion 
piece provides ''A Resource Book for School-Community Decision-Making." 
In South Dakota, on the other hand, state agency staff carry out 
a legislative authorization for community participation by drawing on 
the services on the Rocky Mountain States Consultant Services, Inc., 

to provide technical assistance to local groups in establishing com- 

15 

munlty participation. 

The available ei>idenoej thetij provides some indicators that 
states are 1) utilizing a variety of mechanisms for participation 
in decision-making at the state level; 2) interacting with various 
types of local advisory groups required by legislations and 3} engag- 
ing in service functions with local districts by providing technical 
assistance in the form of programs and materials. 

2. Interstate Activities 

Although the various Titles of ESEA require or provide models for 
various types of participation, the unique characteristics of Section 
505 of Title V suggest a somewhat different role in influencing the 
climate for educational decision-making. This special project section 
of Title V sets aside a percentage of funds to be used: 

To pay part of the cost of experimental projects for develop- 
ing state leadership or for the establishment of special 
services which, in the Judgment of the Commissioner, hold 
promise of making a substantial contribution to the solution 
of problems common to the state education agencies of all or 
several states, and for graivts to public regional Interstate 



from a high of $4.4 million in FY 1968 to well below $1 million since 
1971), several features of the program deserve attention. 

Between 1967 and 1972, 58 interstate projects were operational 
under this provision. The following table indicates the type of par- 
ticipation and project substance: 
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TABLE 1.3 

SECTION 505— INTERSTATE PROJECTS 1966-1972* 



Examples of Types of Projects 

Conferences on information systems, school 
district organization, training for school 
board of education members, training insti- 
tutes for Chief State School Officers 

15 Comprehensive Planning — S^ate leader- 
ship, the future of state education, state 
planning, program coordination 

18 Organizations and Administration — 
Certification of teachers, finance, 
management information systems, management 
of migrant programs 

8 Teacher Training and Curriculum — Demon- 
stration centers, instructional materials, 
selection and evaluation materials and 
equipment, performance-based teacher edu- 
cation 



Compiled from: State Departments of Education and Federal Programs: 
Annual Report. Fiscal Year 1970> DREW Publication 
No*(OE) 72-68, and updated information from USOE Office 
of State Agency Cooperation. 



Patterns of interaction within the forty-one interstate projects 
reveal a variety of relationships. Twenty-two of these projects were 
designated as regional in character and six as non-regional (Table 1.4) • 

TABLE 1.4 

PATTERNS OF RELATIONSHIPS IN 41 INTERSTATE PROJECTS* 



22 Regional Projects ; 

6 within boundaries of a federal region 

9 in conjunction with a neighboring region 

7 in conjunction with several regions 

19 Non-Regional Projects ; 

in conjunction with states scattered through 
the nation 



Compiled from: State Departments of Education and Federal Programs; 

Annual Report Fiscal Year 1970 > DHEW Publication 
No.(OE) 72-68, and updated information from USOE 
Office of State Agency Cooperation. 



Project Participation 
17 Fifty-State Projects 

41 Interstate Projects 
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TABLE 1.5 
INTERSTATE PROJECTS—1966-1972* 



REGION 



II 



■k-k-k 



Compiled from: State 
Departments of Education 
and Federal Programs; 
Annual Report Fiscal 
Year 1970 , DHEW Pub 11 
cation No.(OE) 72--68, 
and updated informa- 
tion from USOE Office 
of State Agency Co- 
operation 

Participation in 41 
Interstate Projects 
(excluding 17 Fifty- 
State Projects) 

Administration of 
58 Title V Projects 
(including 17 Fifty- 
State Projects) 
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STATE PROJECT 

PARTICIPATION 



ADMINISTERING 
STATE*** 



Connecticut 
Maine 

Massachusetts 
New Hampshire 
Rhode Island 
Vermont 



7 
4 
3 
4 
4 
4 



New Jersey 
New York 



6 

12 



1 
1 
1 



1 

6 



T T T 
III 


Delaware 


7 






Maryland 


6 


3 




Pennsylvania 


o 
O 


3 




Virginia 


3 






West Virginia 


8 


1 


IV 


Alabama 


7 


2 




Florida 


14 


3 




Georgia 


7 


3 




Kentucky 


4 


1 




Mississippi 


4 






North Carolina 


8 






South Carolina 


4 






Tennessee 


5 


1 


V 


Illinois 


7 


1 




Indiana 


4 






Michigan 


11 


1 




Ohio 


13 






Minnesota 


10 


2 




Wisconsin 


7 


3 


VI 


Arkansas 


6 






Louisiana 


4 




[ 


Texas 


9 


3 




New Mexico 


4 






Oklahoma 


/ 




VII 


Iowa 


5 


3 




jxansas 








Missouri 


4 






Nebraska 


4 




VIII 


Colorado 


12 


6 




Montana 


4 






North Dakota 


3 






South Dakota 


9 






Wyoming 


4 




IX 


Arizona 


5 






California 


12 


2 




Nevada 


4 






Hawaii 


4 




X 


Alaska 


3 






Idaho 


5 






Oregon 


7 


2 




Utah 


9 


2 




Washington 


11 


1 
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Every state has participated in at least three of the forty-one 
Interstate Projects and nine states have participated In more than 
ten. All fifty-eight projects functioned through an administer- 
ing state, and twenty-six states have played this role at least once} 
nine states have administered three or more projects. (Table 1,5)^^ 

Without attempting to assess the quality of program sub^Jtance, 
the immediate results of projects, or the intricacies of how projects 
and states were chosen it is possible to conclude that this piece of 
federal legislation focused state attention on certain issues ^ such 
as management and planning y and faailitated activity related to these 
issues. It also facilitated cross-estate interaction not restricted 
to geographical locales* This level of interstate involvement sug- 
gests at least the rudiments of an informal multi-faceted interstate 
structure of interpersonal contacts and communication channels both 
within and across regions. 



Recapitulation 

Federal action in the administration of ESEA included: 

1) requirements for information gathering and reporting including 
detailed demographic data and objective measures of Title I student 
performance; 2) general requirements for state plans and funding and 
opportunities for the planning and evaluation of federal programs; 
3) funding and encouragement for comprehensive planning. State 
activity under ESEA has included: 1) significant increases in personnel 

2) expanded assessment activities; 3) the establishment of planning 
units within SEA's. 

It should be emphasized that this type of cooperative interstate 
activity is not at all exclusively related to federal programs, 
although it is reinforced by other federal activities such as the 
Rocky Mountain States Satellite Project. A number of other activ- 
ities such as foundation-supported programs of Carnegie and Ford 
have developed similar cross-state ties. In addition, the wide- 
spread use of management or educational consultants supports other 
networks of related activities across states. 
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Through legislation and guidelines for advisory councils > the 
federal government has established modes of participation in decision- 
making at the federal I state, and local levels. State activity 
demonstrates a variety of patterns of participation with both state- 
level and local-level councils* In addition, interstate projects 
have established formal mechanisms for interstate cooperation both 
within and across regions » 
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PART THREE 



FEDERAL ACTIVITY AND NATIONAL EDUCATIONAL COALITIONS 

The array of education groups impacting on federal policy is 
extensive. Among the most visible are the members of the "Big Six*': 
the American Association of School Administrators, the National Edu- 
cation Association, the Council of Chief State School Office, the 
National School Boards Association, the National Association of State 
Boards of Education, and the National Congress of Parents and Teachers, 
Within NEA, each professional role, such as the elementary and sec- 
ondary school principal, has its organization. Teachers are also 
represented by the powerful American Federation of Teachers, In 1964, 
governors and state education agency personnel formed a "Compact^' 
eventually leading to the Education Commission of the States, As 
described by James Koerner, citizens* groups have been more ephemeral: 
the national Citizens Commission for Public Schools flourished in 
1949, the National Citizens Council for Better Schools in 1956, the 
National Committee for Support of Public Schools in 1962.^^ The list 
of national groups could be expanded almost indefinitely to include 
special interests such as the National Catholic Education Associa- 
tion, testing and accrediting agencies, as well as higher education 
organizations • 

A significant move for the consolidation of power among major 
groups occurred in 1969 with the formation of the Emergency Committee 
for Full Funding, Although the group was successful in pressuring 
Congress to override a 1970 presidential veto of educational appropri- 
ations, the diversity of interests within the coalition has caused a 
cyclical pattern of activity and quiescence since then, 
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The move towards national power groups, however, continues to 
expand In a variety of ways: Increasing momentum among established 
groups, reactivation of dormant groups, formation of new ones. 
Several examples support this view. The National Association of 
Secondary School Principals has discussed proposals to sever affilia- 
tion with NEA and to operate Independently, The National Committee 
for Support of Public Schools has been reactivated under the leader- 
ship of Carl Marburger, J, William Rleux, and Stanley Salett. The 
formation of the Coalition of State, City and County Officials was 
announced In August, 1973 as a lobby **to Increase the Influence of 

non-federal elected officials over federal programs operating In 

18 

their jurisdictions,'* National level action continues unabated, 

The entrance of the federal government Into the arena of ele- 
mentary and secondary education traditionally reserved to the state 
and local agencies produced shock waves In the educational establish- 
ment, - :h activity was perceived as a threat to state and local 
autonomy and It generated strong reaction among groups such as the 
Council of Chief State School Officers, On the other hand, federal 
funds have supported some activities of groups such as the Education 
Commission of the States and the National School Board Association, 
The following overview of recent activities of these selected groups 
suggests that, bc:h by precipitating opposition and by supporting 
activities, ESEA has played a role in solidifying the power of nation- 
wide groups that articulate educational policy at the federal, state, 
and local levels. 

Council of Chief State School Officers (CCSSO) 

Because authority for education is vested in the states, histor- 
ically the structure of state educational activity has been marked 
by isolation and fragmentation among the fifty states. State agencies 
emerged in various states over a hundred-year period, subject to 
widely differing constitutional authority, and with diverse adminis- 
trative styles and relationships with local districts and legisla- 
tors • Over time state agency functions hav3 evolved from simple 



data-gathering activities of the nineteenth century to various types 
of leadership roles*^^ A number of factors contributed to some 
similarity of developments among the states: data-gathering activ- 
ities required by the federal education agency with Its establish- 
ment in 1867; curriculum requirements at the secondary level gener- 
ated by the formalization of college entrance requirements; the 
subtler Influences of mass textbook production; the development of 
national professional groups (AASA, NEA, NASSP); and the market 
monopoly enjoyed by a few prestigious universities in supplying 
school superintendents. These influences came from diverse sources 
and they were to a large extent outside the formal authority struc- 
ture of state departments of education* 

In 19A8, the superintendents of education In each state formed 
the Washington-based Council of Chief State School Officers. In 
broad outline, this group can be seen as an alliance of state educa- 
tion officers functioning in three waysj first, it builds communica- 
tion networks among those holding major offices through scheduled 
conferences, Regular newsletters serve a second major function by 
channeling information to all states, identifying Issues, marshalling 
arguments! enlisting support — a powerful tool to generate at least 
a modicum on consensus among states traditionally perceived as dis- 
crete entitles. Finally, both the Interpersonnel networks and the 
information channels enable the organization to function as a lobby, 
impacting on decision-making at the federal level. 

Perhaps the most striking testimony to the unification develop- 
ing among SEA's is two volumes, representing several years work, 

which articulate CCSSO policy on federal-state and state-local rela- 

21 

tlonshlps. The volumes are essentially a summary of policy positions 
taken over a period of years. The discussion of federal-state rela- 
tionships addresses the issue of federal intrusion into state policy, 
insists on the rights of states to control education within their domain, 
and politely, but strongly, urges appropriate and somewhat vague pat- 
terns of ^'partnership." The existence of such documents is perhaps 
even more significant than their substance — the chief education 
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officers of fifty states speaking as a single voice on national 
educational policy* 

In two major instances since 1965, CCSSO demonstrated Its 
power as a lobbying force capable of Impacting on federal legisla- 
tion, In 1967, CCSSO Joined forces with the ^^Blg Six'' to gain state 
control over 85% of Title III funds. ^ The group was also Instru- 
mental In gett-fng congressional approval for the Title IV Amendment 
which authorized funds for state planning, 

CCSSO regularly publishes policy statements on pertinent issues. 
The first Item of the report of the Annual Business Meeting of 
November 15, 1972, for example, comments on Accountability! 

Accountability 

The Council of Chief State School Officers believes 
that a clear set of goals, and a reporting of the degree 
to which these goals are met, are essential In American 
education. 

Therefore, the Council urges each chief state school 
officer to provide required leadership in the setting of 
goals and the evaluation and assessment of all programs 
Involving student time and public funds, appropriately 
publicizing achievements and deficiencies, so that all 
citizens may know the results of their Investment in 
education. 

The final Item addresses the problem of federal-state velatlon- 
ships: 

USOE-CCSSO Relationships 

The Council of Chief State School Officers views its 
relationship with the Office of Education as more than 
periodic, routine mutual endorsements In principle. The 
Council declares that it Is imperative that there be prior 
formal consultations by USOE with groups of chief state 
school officers formed by the Council before legislative, 
regulatory guidelines or budgetary Initiatives are taken 
by USOE which will significantly affect state education 
Interests • 

The Council acknowledges the concerns of the Congress 
over Office of Education/National Institute of Education- 
State relations In education, and pledges that the Congress 




will be kept fully informed by the Council on the status 
of these relationships,^^ 

In a statement of July 9, 1973, CCSSO expressed strong resistance 
to efforts of the present Administration to decentralize the Office 
of Education by expanding the functions of ten regional offices • 

The character of the USOE-CCSSO relationship has, however, fluc- 
tuated widely over time, shaped by both the political climate and 
the force of central personalities. The groups have joined forces 
in a number of ways designed to strengthen SEA's — summer Institutes 
for Chief State School Officers, institutes for mid-management SEA 
personnel, study seulnars, and special projects such as special edu- 
cation and career education. 



A recent phase of CCSSO activity signals a potential new phase 
of relationships. In January 1973, as a result of CCSSO initiative, 
the Office of Education joined with the group to establish the Com- 
mittee on Evaluation aifid Information Systems (CEIS) . Members of twelve 
sub-committees who are SEA (and LEA) personnel will cooperate with 
liaisons from major OE programs to review OE activities in the areas 
of data-gathering, planning^ and evaluation. Since the groups have 
not yet begun to function fully, the impact of this alliance cannot 
yet be determined. Whatever political interpretations can be assigned 
to this move, from an organizational point of view the committee can 
be construed as structurally meeting a basic organizational principle 
of involving those who will be affected by action. In this sense, 
the establishment of the committee represents a minimal condition 
necessary for program implementation. 

The CCSSO has demonstrated in the past that it can Influence policy 
at the national level. The formation of CEIS Is a major development 
at the federal level that deserves careful attention in terms of its 
impact on the scope of educational activity related to data-gathering, 
planning, and evaluation. The Committee also deserves study in terms 
of the range of participants: the level of involvement, the adequacy 
of representation, and breadth of interest, 
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Education Commission of the States (ECS) 



The Education Commission of the States has also emerged as a 
national coalition of education personnel. Established In 1964 as 
a "Compact** with a special focus on legislative Issues within states 
and with a membership consisting of governors as well as education 
personnel, the organization has a base for substantial political 
Influence. ECS serves as a clearinghouse for Information on activ- 
ities within states. Like CCSSO, It too builds Interpersonal net- 
works and Information channels. Its Impact on decision-making Is 
somewhat more difficult to pinpoint because Its primary target groups 
are within the states. 

Like CCSSO, ECS constitutes a "federation" of state personnel, 
but it functions as a service organization rather than as a policy- 
making group. A regular newsletter announces ECS administrative 
meetings; regional conferences highlight pertinent legislative 
issues at the state level* In addition^ the ECS Legislative Review , 
covering legislative activity in the states, is published weekly 
when legislative activity is at its peak and monthly during the rest 
of the year. 

A major production of ECS Is a periodic compendium of legisla- 
tion in the states. Legislation, Achievements, and Problems in Edu - 
25 

cation. ^ The 1972 document is a veritable encyclopedia of state 
activities, carefully categorized> and systematically identifying 
appropriate contact people in each state agency. It thus not only 
provides information, but it facilitates further contact and sharing 
among states. The 1972 Survey, for example, contains seventy items 
describing state activity in the area of accountability compiled from 
the reports of twenty-eight states. 

There is little to Indicate to what degree such a document is 
utilized by state personnel and, therefore, to assess its impact. 




It is clear that state officials at a variety of levels rather 
assiduously forwarded pertinent information for inclusion in the 
compendium. Given the growing evidence of ways in which states look 
to other states for Information and expertise, the utility and in- 
fluence of the ECS data bank is self-evident, and its potential for 
shaping state activity strong. 

Because ECS functions with an organizational staff, it offers 
an alternative vehicle for carrying out a national program. The 
most striking example of the importance of this capability is the 
transference of the management of the National Assessment of Edu- 
cation Progress to ECS in 1969. The project had evolved since 196A 
through Joint foundation and OE funding. The shift in the locus of 
management responsibility was the culmination of a power struggle 
between OE and the states. National Assessment publications 
report the transfer in the following way: 

The Education Commission of the States assumed full 
responsibility for managing the, National Assessment 
program on July 1, 1969 because of its interest in 
accountability and after it had been requested by 
educational organizations who felt that the Commission 
was broadly based and appropriately responsible to 
the public . ZT^pEmphaaia added] 

This statement raises a number of immediate questions such as: 
To what public is ECS responsible? And somewhat more probing ques- 
tions such as: What is the source of ECS interest in accountability? 
How is this interest communicated to its "bKoad base"? What impact 
does this interest have on shaping educational activities in the 
states? (A current ECS policy, for example, is to encourage and 
facilitate the use of National Assessment at the state level. At 
least fourteen states are currently utilizing National Assessment 
tests and each ECS Bulletin reports the Increasing momentum.) 

The federal role in triggering the growth of ECS is less obvious 
than in the case of CCSSO, but in several ways the structure of the 
organization and its functions are complementary to CCSSO. Like 
CCSSO it stands as a strong and highly visible coordinator of state 
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Interests, Because of its service orientation It could be expected 
that It will continue and expand Its operations. Such development 
would mark an Important escalation of a coalition movement In educa- 
tion and would give new prominence to ways In which ECS Interests 
are determined and the Identification of the publics to which It Is 
responsible. Furthermore, the Initial Indicators that ECS helps to 
shape SEA activity and provide a unifying (and uniforming) Influence 
on states means that the substance of what Is communicated deserves 
careful study and analysis. 

National School Boards Asso<::iatlon (NSBA) 

The organization of the National School Boards Association offers 
sharp contrast to both CCSSO and ECS. Local school board members 
are represented in a state association which in turn elects repre- 
sentatives to the national association. The national organization, 
which was organized in 1940, moved to Washington in 1966. 

The somewhat unwieldy representational structure which includes 

a general assembly was significantly altered in 1971 to allow direct 

membership of local board members. Services to this group Include: 

1) a fortnightly newsletter, 2) workshops and seminars, 3) research* 

The national association directly states that the financial resources made 

available by this program ($250 to $5,000 per district depending on 
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size) will support lobbying activity at the federal level. The 
program also opens up a new channel for providing technical assistance 
for local boards. The educational policies research branch of the 
organization, funded in 1968 by the Office of Education, has produced 
a wide range of manuals, handbooks, kits, and other materials for 
codifying school board policies and updating practices. Approximately 
1700 local school districts presently utilize this service. In 
addition, NSBA offers the American School Board Journal which has a 

H 

Although the National Association of State Boards of Education also 
offers a rich source of material for this discussion, the local 
school board organization has been selected for examination here 
to Indicate momentum at the local as well as the state level. 



subscription list of 50,000 and a variety of resource materials 
including cassettes dealing with problems such as student rights, 
measuring staff performance, community involvement and legal issues. 
Other materials provide guidelines for developing Job descriptions 
in a local district and training packages for problem-solving 
experiences. 

NSBA thus combines its lobbying activity with service functions. 
It is concerned both with providing a voice for the layman in educa- 
tion at the federal level and with "mounting a nationwide long-range 
public information effort" and providing a growing number direct 
services . 



This surge of NSBA activity in recent years has derived fiscal 
support from the Office of Education, The impressive array of 
materials designed to increase the efficiency and effectiveness 
of school boards speaks to major educational issues. The new pat- 
tern of direct affiliate membership signals a new kind of relation- 
ship with local districts and suggests a significant expansion of 
this organization in its capacity to build Interpersonnel networks, 
to channel information, and to impact on decision-making. It is 
a different kind of coalition, with the potential for impacting much 
more directly on local districts, A whole range of questions arise 
from this phenomenon: What is member relationship to SEA's and how 
do these Impact on SEA-local relations and SEA administration? What 
congruence is there between policy recommendations disseminated to 
school boards and state school policies? How effective have been 
previous efforts at technical assistance and what will be the results 
of these new efforts? Is there any coordination between technical 
assistance to school board members and technical assistance to 
school administrators/teachers provided from other sources? Do 
participants fall into any distinctive geographical breakdowns and 
is there any relationship between the patterns of users and other 
educational directions discernible within the state? 



Here, as with ECS, the potential of the organization for 
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Impacting on schooling across the nation is of significant proportions 
to merit systematic and careful analysis* 



Summary 

This description of the emergence of new educational coalitions 

at the national level has necessarily been selective and highly 
29 

generalized. It can, of course, be argued that the phenomenon 
outlined here Is nothing other than a continuation of a pattern of 
the nationalization of Interest groups represented by associations 
such as the American Federation of Teachers, the National Education 
Association, or the American Association of School Administrators. 
The counter-argument suggests that seven years of massive federal 
educational programming have helped to create a substantially dif- 
ferent scene by offering both a challenge to control that solidified 
group interests and a source of support monies that strengthened 
their capabilities. Furthermore, the groups identified here repre- 
sent petfeonnel with assigned authority and responsibility at state 
and local levels. The consolidation of these groups at the national 
level can impact directly on both state and local agencies. 



The focus on the role of the federal agency in this development 
in no way suggests sole or even major influence on the development 
of the coalitions described. From one point of view, the strengthened 
power of sea's is part of a much broader general effort to increase 

the capabilities of the states. Such efforts have been heavily sup- 
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ported both by foundations and by citizen groups. Whatever the 
range of contributing factors, this "nationalization" trend of edu- 
cation is not simply an accidental inevitability, but a direction 
that has been specifically supported by a number of forces. The 
increasing power of educational decision-makers can be variously 
interpreted as 1) a counterforce to face up to demands of pressure 
groups such as AFT and NBA on one hand and the federal government 
on the other; 2) a consolidation of power by yet another professional/ 
political interest group; or 3) a new energy to facilitate systematic 
Improvement of education on a national scale. Given the current 
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flux in the educational scene* It is not yet clear which of these 
Interpretations holds the most validity > and the conclusion Is not 
at this moment Inevitable. 



The nationalization of educational issues through dramatic 
court decisions has set the stage; instant communication possible 
in today's society has helped to set the actors in motion. These 
national groups exist; their capabilities are apparent. They will 
receive challenge, encouragement, support from a variety of direc- 
tions. It is the character of the expectations proposed to them 
from multiple publics, the kinds of demands and the kinds of support 
they receive, that will determine what role they play in the educa- 
tional enterprise in setting directions and priorities. 
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PART FOUR 

SOME I^U^LICATIONS FOR FEDERAL POLICY-MAKERS 



Several movements converge at this time to create new pressures 
on the Araerlcan educational scene. The move to decentralization, 
reflected in discussions for revenue sharing and grant consolidation, 
promises to change patterns of educational decision-making. Shifts 
In the lines of power and responsibility and in the channels for 
resource allocation will focus attention on different levels of the 
governmental hierarchy and stimulate new patterns of interaction and 
cooperation among relevant actors* 

Current administration support for Special Education Revenue 
Sharing has been clearly articulated in public documents which indi- 
cate the purpose and the expectations* The purpose, expressed in 
the proposed Better Schools Act of 1973 , is summa; ized by HEW Secretary 
Caspar Weinberger in the letter of transmittal to House Speaker Carl 
Albert: 

It is the purpose of this bill to consolidate certain 
elementary and secondary education grant programs into 
a system of Federal revenue sharing for education de- 
signed to meet the needs of State and local school sys- 
tems and to do so in a manner designed to provide State 
and local education officials with the flexibility and 
responsibility they need to make meaningful decisions 
in response to the needs of their students * [ Emphas is 
added ] 

In supporting the Bill before the General Education Subcommittee of 
the House of Representatives, Secretary Weinberger stated that the 
Bill would "redefine the Federal role in elementary and secondary 
education" and "remove the Federal straitjacket which assumes that 
what is good for one State is equally beneficial to another." 



In addition » the Bill seeks 

nothing less than a new definition of the relationship 
between the Federal government and State and local 
governments — one which responds to present educational 
needs and anticipates the needs of the future. 

Neither the letter of transmittal nor the statement of Secretary 
Weinberger, however, suggests the precise character of this rela- 
tionship, appropriate measures of responsiveness or procedures fot 
anticipating future needs, 

The Administration's decision to withdraw the Better Schools 
Act does not signal a retreat from its basic principle — the devolu- 
tion of power (and problems) to the states. Plans for education 
revenue sharing and grant consolidation which reflect the principle 
are very much alive and supported by representatives of the educa- 
tional community at a variety of levels* Furthermore, current em- 
phasis on the state role in education receives additional impetus 
from the surge of so-called accountability legislation in the states 
which is documented by Maureen Webster in the following chapter. 
Shifts in state roles are occurring in practice as well as in 
rhetoric. 

The central function of recent federal legislation has been to 
channel funds to meet educational needs as perceived at the national 
level. The proliferation of categorical grants, however, has pro- 
duced restrictions on the use of funds and fostered uncoordinated 
administrative patterns at state and local levels which are now 
recognized as dysfunctional. Revenue sharing and grant consolida- 
tion are policies designed to reduce this restrictiveness and to 
allow greater flexibility at the state level. In the process, how- 
ever, federal responsibility In the federal-state partnership is 
becoming defined ever-more exclusively in terms of fiscal support 
only. Although administration policies do point to plans for tech- 
nical assistance to states, the concept is vague and the commitment 
unclear. 
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The previous sections of this paper have pointed to three factors 
pertinent to decisions which reshape the federal-state partnership: 

1* the current expansion of assessment and planning 
activities at the state level { 

2. the Increasing activity of national coalitions 
Influencing decision-making at federal, state, 
and local levels; 

3. the untapped resource of recent federal experience 
and Its Influence on state and local policy. 

Each of these features of the educational scene has significant im- 
plications for a federal agency. 

Expansion of Planning^ Assessment > and Participation 
at the State Level 

The burgeoning state activity in planning, evaluation, and par- 
ticipation outlined in the preceding section will be described in 
detail in the following chapter, The extended analysis highlights 
both common threads of the state experience and patterns of variation 
among states. Furthermore, it hints that intensive activity in indi- 
vidual states is building a body of experiences that characterizes 
certain states as ''lead" states. 

It is clear that much of this new activity, especially in the 
area of planning and assessment, has been stimulated, supported, and 
sometimes Justified by federal funds. These changes currently under- 
way in state agencies will stabilize. Increase in momentum, change 
direction, or cease. They will be affected in yet undetermined ways 
by changes in funding patterns, 

Moves towards decentralization give Increased importance to such 
activities in state agencies. The federal agency, therefore, has a 
stake In developments at the state level In the consequences of revenue 
sharing for states and for their capacities to plan, assess, and guide 
it. This means, at a minimum, tracking the patterns of changes which 
occur in agencies subsequent to funding shifts in order to have 



solid Information about the consequences which can guide future 
action. It could» In addition, mean the development of new types 
of technical assistance programs to distill the common experience 
of states 80 It can be shared, to facilitate problem-solving activi- 
ties for those situations unique to a state, and to consolidate the 
gains apparent In "lead" states by increasing communication among 
states. 

In addition, the expanding role of state agencies and Increased 
pressures for accountability are pushing states to clarify their own 
roles and responsibilities In relation to local districts. A care- 
ful analysis of federal-state Interactions could produce analytic 
tools which raise the Issues that states, too, must deal with in 
assessing the character and quality of state-local relationship. 

If the ouvvent shifts in funding patterns ave to provide the 
base for a redefinition of federal-state relationships^ there is 
need to identify concurrent changes in state agencies^ and to antici^ 
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pate the consequences of those changes for federal policy. 

The Increasing Activity of National Coailtlons 

The proliferation of educational Interest groups at the national 
level and the increasing power of those already functioning, espe- 
cially those representative of educational policy-makers, indicates 
a trend that is likely to Increase rather than diminish. Thr* phe- 
nomenon of national coalitions has reached a point where it is pos- 
sible to distinguish, at least conceptually, between federal educa- 
tional policy which guides the activity of the federal government and 
national educational policy positions which represent a wide array of 
concerns of Interest groups and decision-makers. Recent years of 
federal activity in education have been marked by cooperation, hos- 
tility, and compromise between these two. The increasing power of 
coalition** suggests that in the immediate future patterns of relation- 
ships will solidify. The emerging pattern will either sharply limit 
the federal role or it will creatively shape new styles of interaction 
between federal and "national" groups. 



Because coalitions have their own momentum generated by their 
constituencies, the main responsibility for the character of the 
relationship rests with the federal agency. At present, there 
appears to be no clear and consistent policy guiding federal rela- 
tionships with various groups, The conceptual and practical prob- 
lens related to formulating such a policy are, of course, enormous, 
but the federal agency will Inevitably be faced with decisions to 
Interact with or to Ignore these groups. Analysis which surfaces 
the problems and explores alternative policies can provide a ground- 
ing for such decisions. 

The inoveasing visibility and power of national interest groups 
points^ therefore^ to the need to address the issues they raise for 
a federal agency^ to articulate the rationale for current practice^ and 
to explore and assess alternative policies. 

The Untapped Resource of Federal Experience 

Since the NDEA legislation of 1958, the U.S» Office of Educa- 
tion has not only channelled funds for targeted educational programs; 
It has also Initiated and supervised extensive activity In teacher- 
training, curriculum development, planning and evaluation, partici- 
patory mechanisms. Institutional organization and administration. 
There Is, therefore, more than a decade of accumulated experience 
derived from the successes, failures, and ambiguities of activity 
In every phase of the educational enterprise. At some point In time, 
each one of these areas has received attention as the "key*' to edu- 
cational changes and funds have flowed first to programs and later 
to populations to achieve this goal. (In this context, programs 
for decentralization ran be seen as the most recent strategy to solve 
the problems of education or as a rejection of the notion of a federal 
solution.) 

The mode of accountability required by Congress and the mandates 
of specific pieces of legislation have resulted In a federal evalua- 
tion policy focused on discrete programs and projects. There has 



been neither mandate nor mechanism to direct attention to broad 
questions of federal Influence. The narrow focus of Individual 
pieces of research provides only limited perspectives on the large 
picture of federal action.* 

Analyses of Individual programs do Influence policy, but the 
view can often be short-term and the base of Information narrow. 
Reliance on this mode of research alone contributes to a style of 
decision-making which draws on Insufficient Information to terminate 
apparently unsuccessful programs and to move quickly to new strategies. 

Because the federal government has now been Involved In elementary 
and secondary education for eight years, there Is a backlog of ex- 
perience that lends Itself to broad-based analysis. Besides expanding 
the base for federal decision-making, such analysis could also be use- 
ful at the state level. Added decision-making power at the state level 
means Increased scope of action for states as well as Increased polit- 
ical pressure. At this moment, some states are searching for levers 
for change, selecting from a variety of strategies for action: teacher 
training, curriculum, planning and evaluation, organization and manage- 
ment, public participation. Although so e states will forge ahead to 
mold new and creative strategies, others will simply repeat the cycle 
of federal history, moving from one to the other in search of the 
"key.'' Even though the situation of the states and the Office of 
Education are not entirely analogous, there is an accumulation of 
experience at the federal level in addressing problems of strategies 
for change that coCild inform decision-making at the state level. 
Such analysis could Identify the educational Issues common at all 
levels. It could also begin to sort out the organizational and 
political Issues unique to each level of decision-making. Such 
research offers a base for a new type of federal-state interface by 
providing a special form of technical assistance. 

* T' 

Jerome T* Murphy's examination of the effect of Title V funds on 

state agencies, for example, draws conclusions based on evidence 

in three states and excludes interstate activities funded under 

Section 505 and planning activities funded for the first time in 

1970. 
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There is, then, a need for studies which review and synthesize 
federal experience with change strategies and which translate this 
into policy options for federal action and into policy analysis for 
state use. 

The new legal and influence structures described In previous 
sections are creating a situation of fluidity and uncertainty. The 
burden of this presentation has been to point out that^ both directly 
and Indirectly, the federal apparatus (Including both legislative and 
administrative branches) has helped to shape the current educational 
scene* Through Its own operating style» through fiscal resources, 
and through administrative guidelines, It has supported the develop- 
ment of new structures for educational decision-making. It Is through 
such structures that the future of education will be Invented, The 
quality of that future depends upon the care with which these struc- 
tures are examined and the soundness of the decisions they generate. 



ACTION AND ACTORS ON THE NATIONAL SCENE 
by Miriam Clasby 
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fifteen years of educational change* Efforts to provide some 
kind of integrated or comprehensive perspective usually produce 
a collection of independent and discrete analyses of various 
aspects of the educational scene* See, for example, Emerging 
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THE QUEST FOR BETTER SCHOOLS 

Statewide Testing Legislation 
and Educational Policy 

by 

Maureen MacDonald Webster 



PART ONE 

LEGISLATION^ ACCOUNTABILITY > AND STATEWIDE TESTING 
Introduction and Overview 



The Changing Context for Decision-Making 

Concerns for social Justice, for quality^ for efficient and 
effective management In the domain of education are not new. However, 
the contemporary translation of these concerns through pressures for 
"accountability*' for the outcomes of schooling In relation to expendi- 
tures of tax dollars Is a new phenomenon. The push for accountability 
is growing and it is creating phanges in the context for decisions 
about educational policy. This development has its source in several 
trends of the past decade* 

- The provision of virtually universal public schooling 
through high school allows and impels an increasing shift 
in attention from responsibility for access to schooling 
to responsibility for its outcomes. 

- Massive federal involvement in education has contributed 
to rising expectations and increasing awareness of dis- 
advantage on the part of those with sub-standard schooling. 

- The ambivalence of several major publicized evaluations 
of the Impact of massive subsidization has eroded con- 
fidence that large infusions of tax dollars will result 
in substantial improvements in the measured performance 
of schooling* 

- The costs of schooling are increasing together with the 
resistance of taxpayers to the rising curve of expendi- 
tures on education. 
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TABLE 2.1 

STATE LEGISLATION FOR ACCOUNTABILITY IN EDUCATION 
Through Fall 1372 



A. Legislation introduced and enacted 




23 states' 


Introduced 


29 states' 




Enacted 


23 states 




Not enacted 


estates 




B. Legislation may be Introduced In 1973 




10 states 


May be introduced 


16 states 




States included in A 


6 states 




States not included In A 


10 states 




C. No legislation enacted or proposed 




18 states 


No enactment to fall 1972 


28 states 




Possible In 1973 


10 states 





Notetr 1« Includmf the Ofttrlct of Columbia 

2. tncludmgtwoitatet where other accountsbitity legiilatton wai enacted 

Source: Derived from information reported by the Cooperative Accountability Project^ 
May and October 1972, and April 1d73 



Webiter/EPRC 
May 1973 



State Accountability Legislation Circa 1972 



Many states have responded to these trends by legislation. By 
Fall 1972 accountability legislation had been introduced in twenty- 
nine states and enacted in twenty-three • Bills will probably be 
introduced in ten more states during the 1973 legislative sessions.* 

Some fifty-four specific pieces of legislation, recorded to 
date in the Cooperative Accountability Project, were studied for 
this analysis.'*' Thirty-four of these are dated 1971 or 1972 and 
twelve are dated .1969 or 1970. Thus, over 80% of the legislation 
was introduced in the past four years. 



Table 2.4 (pages 66-67) shows the emphases of legislation in 
the twenty-three states with accountability statutes. 

- Thirteen states require statewide testing/assessment 
of student performance. Three of the statutes Include 
a requirement to develop a uniform information system 
and one includes provision for evaluation of profes- 
sional employees. 

- Twelve states require the development of management 
information systems (PPBS, MIS» etc.). In three cases 
these requirements are included in a statute that pro- 
vides primarily for testing/assessment* 

- Eight states require the development of systematic 
procedures for evaluating the performance of profes- 
sional employees. This provision Is combined with 
student assessment requirements in one statute (for 
Virginia). 

- One state has enacted legislation to regulate the use 
of performance contracting. This is California's Guar- 
anteed Learning Achievement Act of 1971* 



See Table 2.1, page 64 « 



The statutes and their status are detailed! in Tables 2*2 and 2.3 
in the Appendix, pages 202-204. 
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TABLE 2.4 



PATTERNS OF LEGISLATION FOR ACCOUNTABILITY IN EDUCATION 

Through Fall 1972 



State 


lYianagBmeni 
Systems 


lesiing ( 
Assessment 


Employee 
Evaluation 


Performance 
Contracting 


Alaska* 


PPBS 1970 






Illinois 


PPBS 1972 






Indiana 


PPBS 1971 






Hawaii 


PPBS 1970 






Ohio 








New Mexico 


UAS1967 






Ari20na* 


UAS 1972 


1969, 1972 




Nebraska * 


MIS/UAS '69 


1969 




Rhode Island* 


MIS /UAS '69 


1963, 1969 




Colorado 


PPBS 1971 


1971 




Pennsylvania 




1963 




Massachusetts 




1965 






Michigan 




1970 




Wisconsin 




1972 




Maryland 




1972 




Connecticut* 




1971 


1972 




Virginia* 




1972 


1972 




Florida 


1968, 1970 


1970, 1971 


1969 




California 


1967, 1971 


1965, 69,71 


1969, 1971 


1971 


New Jersey* 




[19721 


1971 




Oregon 






1971 




South Dakota 






1969, 1971 




Washington 






1969 




Number of states 


12 


13 


8 


1 



Key : PPBS ■ Planning-pfogrammlng-bodgeting system 

UAS Unilorm Accounting system 

MIS " Management information system 

f*^ ^ Pattern of statewide testing and associated legislation 



TABLE 2.4 continued 



•Notes 

♦Lejiiilatlon for Arliona, Nebraska, Rhode Island, end Viffllnia contains more than one component of 
accountablMty In the same bill, 

•Alaska: PPB$ (s required for all state aoencles under 1970 statute. 

•Connecticut; The 1972 statute requires the development of e personnel evaluation prcyam; I.e., It 
does not mandate Implementation of an evaluation program. 

•Celiforn/e: The 1967 and 1971 statotet for management systems established committees to study the 
f>PBS area and make recommendations to the State Board. They did NOT mandate PPBS. 

•New Jersey: The 1972 bill did not pass. The Cooperative Accountability Project reports that the bill 
will not be re^presented because "most of the features of the bill are In the process of being carried 
out/' 



Source: Analysis of statutes the texts of which were reproduced by the Cooperative Accountability 
Project, 1972 and 1973, plus additional information from Florida SEA. 
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Statewide Testing/Assessment Legislation 

The largest category of accountability statutes provides for 
statewide testing or assessment of student performance In order to 
evaluate the adequacy^ efficiency^ and quality of education. This 
legislation differs somewhat from state to state and Is open In 
varying degrees to Interpretation by Implementing agencies. It 
affects major dimensions of accountability: the establishment of 
goals, objectives, and priorities; the evaluation of progress towards 
achievement of designated priority objectives; how evaluation Is to 
be used to guide declslon-maklngj and the involvement of various 
publics In decisions about priorities^ evaluation, and Its uses* 
In short, the legislation suggests who Is to be accountable to whom, 
for what, and by \fhat means. 

Table 2.5 details the range of questions posed In analyzing state- 
wide testlvtg/assessment legislation and supplementary Information. 
It Is not, of course, claimed that the legislation and supplementary 
information are sufficftmt to furnish full answers to such questions* 
But the analyses of answers to these questions do reveal particular 
characteristics of current state activities that will strongly affect 
public policies for education. 



Objectives, Limitations and Outline 
of the Ensuing Discussion 

Objectives 

The objectives of the following analysis and discussion are (1) 
to highlight major characteristics of contemporary legislation deal- 
ing with statewide testing and assessment of student performance; 



'Assessment,^ 'evaluation,' and 'testing' are associated in much 
of the legislation. While 'assessment* may be considered the 
broader term, it is noted that In practice achievement testing 
is the major or the only evaluation instrument used in many assess- 
ment programs. In Part IV, below, derinitional distinctions of 
tha terms are presented for purposes of analyzing components of 
the policy-planning process. 
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TABLE 2.6 



QUESTIONS POSED IN ANALYZINQ STATEWIDE TESTING / ASSESSMENT LEGISLATION 

AND RELATED DOCUMENTATION 



Focul 


Questions 


Purposes and 
Intentions 

» ■ ■ ' - ■ ■ ■ 


*-What are the stated purposes/intentions of the legislation? 
What is the rationale for requiring testing/assessment? 


Requirements/ 
Objectives 


-What Is the specific mandate/requirement legislated to 
translate the intentions? Where general goals are stated, 
are specific operational obJectlMs left open/prescribed? 
If they are set forth, what are they? 


Targets of 
Evaluation 


--What is evaluated, in what target groups, and b^/ what means? 
What is to be tested? Left open/specified? What domains: 
cognitive, non-cognitive, . .? Limited topics or comprehensive 
testing? 

-What is the target population? All/some districts? AM/some 
schools? All/some grades? Nature of sampling? 


Means of 
Evaluation 

Information 
Generated 

Information 
analysis and 
interpretation 

Information 
dissemination 
and use 


-What kinds of tests are to be used? Specified/unspecified? 
Norm referenced/crtterion referenced? Verbal/written/ 
behavior-related? National Assessment-related? 

-What information is generated? Test score data? In what form? 
(distrlct/school/dass/teacher/student-related?} 

-What non-test information is generated? In what form? What 
other variables are required by law? Collected in practice? 

-Who collects? analyzes? interprets the information? 
Specified/not specified In statute? What happens In 
practice? 

-Who has access to the information? In what form? Under law? 
In practice? 

-Who Is expected to use the data? For what decision making 
purposes? Are these specif led by law? Implied? 


Participants In 
legislative and 
legislated 
processes 


-Who gets involved, at what level. In the process of initiating 
legislation? in evaiudtlon-reiated activities? in decisions on 
the kinds of questions raised above? 

-Where do funds come from for the specified activities? How 
"tied" are the funds? How continuous? 


Responses and 
Problems 


-What has been the response/reaction of various interest 
groups to programs and outcomes? Are there specific 
indicators of response? 

-What problems and issues pertinent to educational policy 
can be identified? 
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and (2) to Identify related problems and issues of consequence for 
educational policy-planning. 

Limitations 

The limits of these objectives are worth noting. First, the 
analysis deals with specific legislation} it is not an analysis of 
other accountability activities in the states. Second, the iden- 
tification and discussion of problems pertinent to educational policy 
in the remainder of the paper should not be viewed as policy analysis. 
Some elaboration of these limits may be helpful. 

Accountability and Legislation , It cannot be assumed that there 
is no accountability activity in states without accountability legis- 
lation* Information about such activities is important. It would 
help us to understand whether and how effective policies for accounta- 
bility can be created without legislation, Such information might 
increase our understanding of alternative responses to demands for 
accountability in education. That agenda, however, lies beyond the 
^ objectives of this paper. 

Legislation and Interpi'etation , Any full treatment of account- 
ability activities would also examine the interpretive role of agencies 
implementing legislation* The effects of legislation will differ depend- 
ing on who gets involved In the translation of legislation into practice, 
on what grounds, at what level, and in what dimensions. These questions, 
however, cannot be answered within the limits of information analyzed 
in this pap(ir, A focus upon legislation is instructive* but it will 
not capture meanings attached to mandates when they are implemented. 
For example, the short, broad mandate of the 1965 Willis-Harrington 
Act in Massachusetts requires the Commissioner of Education "to assess 
the conditions and efficiency of public and other schools throughout 
the Commonwealth." This mandate has been Interpreted by the Massa- 
chusetts State Board of Education in a way that emphasizes the develop- 
ment, with community participation, of state and local goals for educa- 
tion, together with a design for assessment and evaluation that requires 
eventual identification of performance objectives to measure pupil 
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achievement. This Interpretation encompasses the major elements 

3 

attributed to comprehensive accountability and assessment plans: 
attention to goals, to priorities, to performance objectives, and 
to community Involvement. 

Policy Analysis and Pre**Pollcy Analysis . The following discus- 
sion is limited in another wayi It does not undertake analysis of 
alternative policies. There is no Intent to array the problems in 
any priority order, or to assess which could or should be amenable 
to policy intervention, or to analyze policy alternatives related 
to any single problem. On the other hand, this paper does lay some 
of the necessary base for those activities. 

Outline of Parts II, 111, IV 

Part II, Patterns in Statewide TestinR Legislation , examines 
dimensions of accountability by analyzing the legislation to show 
patterns in purposes and intentions, in administration and participa- 
tion, In collection and management of evaluation information, and 
in the uses of that information in decision-making. 

Parts III and IV move beyond the legislation to examine problems 
and processes. Part III, From Evaluation to Decision-Making , iden- 
tifies, illustrates, and discusses problems in utilizing test-related 
data in decision-making. 

Part IV, The Policy-Planning Process » is exploratory. It dis- 
cusses some dimensions of emerging policy-planning processes in rela- 
tion to the span of action and inquiry of agencies implementing 
accountability legislation, and the range of actors involved in the 
process. 
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Poliay Relevanoe 



Contmpomvy legislation for statewide testing and assessment 
in the states is important to educational policy --mkers at all 
levels of government because it both translates and conditions 
public policies for social justice in and through education — equal 
educational opportunity. Those policies have two major^ inter- 
dependent dimensions: determining and assessing the nature of 
the desired opportunity^-the quality of educational experience 
and outcomes sought for all students; and determining the patterns 
in educational finance and expenditures which will maximize the 
chances of all students achieving that quality of educational 
experience and outcomes* The eecondj which requires designing 
policies for marshalling and allocating public resources^ has no 
meaning without the firsts which defines the purposes or ends to 
be served by fiscal management. The legislation analyzed and 
discussed in this paper deals with those purposes* It is creating 
legal structures in which to ask and answer the questions: equal 
opportunity for what educational experience and achievement? 
and how shall these be evaluated? Therein lies its policy 
relevance. 
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PART TWO 

PATTERNS IN STATEWIDE TESTING/ASSESSMENT LEGISLATION 



An Analysis 



This examination of statewide testing/assessment legislation 
Identifies, documents and analyzes major patterns across states as 
reflected In A) stated Purposes and Intentions, and In provisions 
for B) Administration and Participation, C) Information Collection 
and Handling, and D) the Uses of Evaluation Information. 



Efficiency and Effectiveness 

All of the legislation examined requires some form of evaluation 
of student performance* Either In rather general terms or In varying 
degrees of specificity the legislation deals with what is to be evalu- 
ated, at what level, with what target groups, when, and how. But — 
evaluation for what purposes? 

Testing in education is no new phenomenon. Many states have a 
long history of sponsoring testing programs. Prior to the legislation 
examined here, however, most testing programs were intended primarily 
for the guidance of students. Contemporary accountability legisla- 
tion emphasizes a different primary function for testing programs: 
furnishing information for' state-level decisions about the "adequacy," 



The findings of the analysis are summarized in a series of tables 
Interspersed with the text and are recapitulated on pages 102-104, 
below. Illustrative material is generally indented. 



A. Patterns in Purposes and Intentions 
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*'ef ficiency^^' '^effectiveness,'^ and ^'accountability" of public school- 
ing systems* 

The most frequently mentioned purposes for legislating state- 
wide testing procedures are of this orders 

- to measure objectively the adequacy and efficiency of 
educational programs 

- to evaluate the effectiveness of public schools 

- to analyze costs and differential effectiveness of 
educational programs. 

Thus a primary intent of recent legislation Is to assess performance 
outcomes of schooling statewide in relation to the public resources 
going into education. The information generated by evaluation pro- 
cedures is intended to serve the purpose of informing the public or 
its representatives of what has been achieved in return for expendi- 
tures of their tax dollars. It is in this sense that the statutes 
are concerned with accounting and accountability. The information Is 
also intended to feed into a management information pool as a resource 
for decision-making affecting the future of education* 

What kinds of decisions are to be taken on the basis of the 
evaluation information? Some statutes suggest that evaluation is to 
furnish a basis for allocating state funds to Improve the schooling 
system, to increase performance levels, to raise the quality of edu- 
cation and the life chances of students. But — how are we to translate 
"quality of education" for evaluation and decision-making purposes? 
To begin to answer such questions we need information about goals and 
objectives and priorities among them. 

Goals and Objectives 

In all the statutes examined the statements of legislative pur- 
poses include concern for efficiency and effectiveness in education. 
Two patterns are distinguishable. In some cases goals and objectives 
are "given." In others they are to be generated under the legisla- 
tion. These patterns are referred to as Pattern A and Pattern B in 
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TAPLE 2.6 



PATTERNS IN STATEWIDE TiSTINQ / ASSESSMENT LEQISLATtON 
PURPOSES AND INTENTIONS 



Patttrn A: EfflclancY and Effactivenatt 



ARIZONA 
1972 


"davalop, attablith, and direct the implementation of a continuoui 
uniform evaluation tvttem of dudII achlavaments In rAUtInn to 
meaturabia performance oblectivet In basic subjects." (Ch. 168, 
S.B. 1294, 24h 


CALIFORNIA 
1972 


''to determine the ef fectiveneis of tchoof districts and schools 
in assisting pupils to master the fundamental educational skills 
toward which instruction is directed . . .10 that the ienlsla- 
ture and individual school districts may allocate educational 
resources fn a manner to assure the maximum educational oppor- 
tunity for all dudIIs ..." (Sn 12821 1d72 •« raoaalind IMQ 
version) 


CONNECTICUT 

1071 


"to develop an evaluation and assessment procedure designed to 
measure oblectlvelv the adeauacv and efficiency tha educA- 
tional programs offered by the public schools'^ (P.A. 665. S.I L 


MASSACHUSETTS 
1966 


Requests the Commissioner of Education to assess the conditions 
and efficiency of public and other schools in the Commonwealth. 


NEBRASKA 
1969 


Requires the Department of Education to 'Institute a statewide 
system of testing to determine the degree of achievenoent and 
accomplishment of ali the students within the state's school 
systems, if it determines that such testing would be advisable." 
(B. 959, 6 (d)) 


PENNSYLVANIA 
1963 


"Develop an evaluation procedure designed to measure oblectlvely 
the adequacy and efficiency of the educational programs offered by 
the public jchooJs of the Commonwealth." (1963 Act, S. 290.1), 


WISCONSIN 
1971 


"Develop an educational assessment program to measure objectively 
the adequacy and efficiency of educational programs offered by 
public schools in this state." (Ch. 126, Laws of 1971, 
8.443.115.28(10)), 


[NEW JERSEY] 
1971 


(Bl LL DID NOT PASS. Bill required the Commissioner to 'Inquire 
into and ascertain the thoroughness and efficiency of operation 
of the schools of the Dublic school system of the Stata" 
(A.6.822)]« 



TABLE 2.6 continued 



Pattern 8: Efficiency and Effectiveness 
with Search for Goals and Objectives 



COLORADO 
1971 



FLORIDA 
1968 
1971 



MICHIGAN 
1970 



MARYLAND 
1972 



RHODE ISLAND 
1969 



"Institute en accountability program to define and measure quality 
In education , and thus to help the public schools of Coloi'ddo 
to achieve such quality and to expand the life opportunities and 
options of studenti of this state; furtheri to provide to local 
school boards assistance In helping their school patrons to ck ter- 
mine the relative value of their tchoot program compared to Its 
cost". The program developed is '' to rneasure adequacy and efficiency 

of the educational programs » . 

and specific performance objectives '' fArtlcfe4l, 



begin by di^ve^oplng broad goals 



"To provide for the establishment of educational accountability In 
the public education system of Florida; to assure that education 
programs operated in the public schools of ftorida lead to the 
attainment of established oblectives for education; to provide 
Information for accurate analysis of the costs associated with public 
education programs; and to provide Information for an analysis of the 
cos ts end the differential effectiveness of instructional progrems ." 
{KB. 894, S.2)« Assessn)ent is In ihe context of the 1968 legisla- 
tion for Educational Renewal (S. 229.5511: a process whereby 
goals and objectives of education are continuously modified . . . 

Requires a program to "establish meaningful achievement goals in 
the basic sidlfs . . . provide the state with the Information needed tg 
allocate state funds and professional services in a manner best 



g pr 

calculated to equalize educational opportunities for students to 
achieve competence In such basic skills . . ^ Develop a system for 
educational self -renewal that would continuously evaluate the pro- 
grams ..." (EHB 3886, S.I). 

"To provide for the establishment of educational accountability in 
the public education system of Maryland, to assure thst educational 
programs operated In the public schools of Maryland lead to the 
attainment of established objectives for education, to provide 
information for accurate analysis of the costs associated with puWic 
education programs, bq^ to provide information for an analysis of the 
differential effectiveness of instructional programs , . /' Requires 
the State Board of Education to assist local schoof boards and school 
systems "in developing and Implementing educational goals and 
objectives for subject areas ..." Each school Is to establish 
"project goals and objectives" in line with those of Local and 
State Boards. (SB. 166). 

Requires the Board of Regents for education to establish " a master 
plan defining broad goals and objectives for all levels of educa- 
tion in the state: etenf>entary, secondary andhlj^er," 



Soufce: Derived from texti ri statutes and bills recorded in the Cooperative Accountability Project to Apnl «9/3. Emphaset 
added. 
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Table 2.6 which presents several statements of legislative Intent 
and Illustrates similarities and varieties In purposes and In the 
language that expresses them. 

Pattern A statutes require evaluation of objectives without 
referring to processes or mechanisms for determining goals and ob- 
jectives. The legislation, by itself, in Pattern A is insufficient 
to suggest how goals are generated, priorities determined, and the 
translation Is made to operational objectives. 

Pattern B statutes specify or imply a process wherein goals 
and objectives are examined as an integral part of planning and 
evaluation — of which testing is a component. Thus: 

- MICHIGAN requires assessment in education including 
the establishment of meaningful goals and objectives 

- MARYLAND requires that local school boards develop 
and implement educational goals and objectives 

- COLORADO directs the State Board of Education to 
provide a procedure for the continuous examination 
and improvement of goals for education^ 

- FLORIDA requires establishment of statewide objec- 
tives for education. The Florida strategy for 
Educational Renewal requires a process whereby goals 
and objectives of education are continually modified 
to meet changing needs of clients, 

Performance Objectives and Testing Priorities 

Vfhich outcomes of schooling are accorded priority in any 
given state or community? None of the statutes specifically requires 
or refers to priority ranking of goals and derived objectives. How- 
ever, some insights into priorities can he derived from examining 
the targets of mandatory statewide testing. 

Table 2,7 identifies testing targets specified in the statutes. 
Some states have enacted legislation which would, in principle, permit 
but not require testing of any dimensions of activities in education 
(e.g., Nebraska)} others mandate testing In limited, prescribed areas 
(e.g., Arizona); others specify a wide range of testing targets (e.g., 
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TABLE 2,7 



PATTERNS IN STATEWIDE TESTING / ASSESSMENT LEQISlATfON 
SPECIFICATION OF MINIMUM REQUIREMENTS 



State and data 


Dimensions Tested 


Target Qroups 


of itatuta 


RRR 


Content 
Areas 


(grades) 


California, 1972 ^ 
Florida, 1970+71 
Pennsylvania, 1903 


S 
S 
S 


S + 0 
S + 0 
S + 0 


all 
all 
all 


Michigan, 1970 
^ Wisconsin, 1971 
Maryland, 1972 
Ariiona, 1969 + 72 


S 

S 
S 
S 


1 o o o 


4 + 7 
"several" 
X 
3 


Rhodo Island, 1963 + 69* 
Virginia, 1972* 
Colorado, 1971 
Nebraska, 1969 
Connecticut, 1971 
Maisachusetti, 1965 


« 
« 

X 
X 

X 
X 


• 
* 

X 
X 
X 
X 


el. + ijcy. 
ImpllH all 

X 

X 

X 

X 



Nomi RRR « n^difig, mWnq, eompuUtionat tkHtt 

S • ipacified in itatutt; 0 « txplicitiy Mt open 
X • Implicitly toft opao 

* Rhode Isltnd rtQuirai uniform aptitude and inteHigence tettinp 

* Virginia requtrei teachers to provide varying learning experiences and different achlevennent lUndardi 
according to individual differences in abilities and / or past achievement. 

Source; Derived from examinatJon of reg^tlation reported by the Cooperetive Accountabitity Project in W2 
and April 1973. 
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California); still others require testing In a limited domain and 
permit it at the discretion of the implementing agency beyond that 
domain (e«g*i Maryland)* 

When testing targets are specified, the heaviest emphasis falls 
upon reading, writing and computation* Beyond the traditional ''three 
Rs^^ reference Is made to "content'* or "subject*' areas. This priority 
to cognitive skills In testing programs Is confirmed In the ETS sur- 
vey of State Educational Assessment Programs! 

The content of most current state testing programs-- 
whether mandated or unmandated by legislative bodies — 
is often less surprising than it is significant. The 
states engaged in some form of assessment-by-testing 
are mainly concerned with how well their educational 
systems are succeeding in imparting basic skills.^ 

In practice, a few states are seeking to assess some dimensions 
of non-cognitive development* and several statutes allow consider- 
able flexibility of interpretation. However, there is a wide gap 
between the highly generalized and idealized sets of "goals for edu- 
cation" generated In several states and the specific elements of 
performance subjected to statewide evaluation In practice. Both 
contemporary statutory preeariptiona and information on implemented 
testing programs indicate strongly that '^quality of education^' is 
primarily assessed by measuring performance in **basio*' cognitive 
skills. 



B. Patterns and Trends in Administration and Participation 

The Testing Legislation 

Table 2.8 focuses on statutory provisions affecting control 
and responsibility for evaluation programs, and participation by 
various publics in Interpreting and implementing the legislation. 

In all the statutes there is a clear emphasis on the state educa* 
tton agency as the locus of control and rosponslbillty for Implomonting 

* SCO paRos 86'- 89 1 boloi^. 
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TABLE 2.8 



PATTERNS IN STATEWIDE TESTING / ASSESSMENT LEGISLATION 
ADMINISTRATION AND PARTICIPATION 



Stat* 



Colorado, 1971 
Maryland, 1972 
Virginia, 1972 
Arizona, 1969 + 72 
Michigan, 1970 
California, 1972 + 
Mauachutatts, 1966 
Nebraika, 1969 
Rhode Island, 1963 + 69 
Connecticut, 1971 
Florida, 1970 + 71 
Panntylvanta, 1963 
Wiicontin, 1971 



X 
X 
X 
X 
X 
X 
X 
X 
X 
X 
X 
X 
X 



X 
X 
X 
X 
X 
X 



X 
X 
X 



ixi 



X 

(XI 

ixi 



X 
X 



Not*: 'X • Provttion fw (ormtt »clvltory group, iX) • othtf ryp«< of provitlon ftx (Ufticipitlon. 
Sm eomni«nl»r¥ in t««l (<x variMiooi. 



Source Intof milKMi in iui«wic<* (ttting l*gi|l*tion rtpoittd by th* Cooperativt Accountability 
Pioifctin 1972 M'ld Aptil 1973. 
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testing/assessment legislation* Even wherep as In Colorado and Mary** 
landi there Is extensive provision for developing local accountability 
plans* some control rests with the state agencies. Technical assist-- 
ance from state to local agencies Is required by statutes In six 
states) advisory groups ^re mandated In two states} other fotms of 
participation are specified In three more; and contracting services 
are authorized In two states* In practicep these elements occur 
more generally than the legislation suggests* Thus, technical 
assistance of state to local agencies » associated with various 
accountablllty-plannlng-evaluatlon activities^ is an expanding function 
In many states. Agaln^ agencj.es implementing testing and assessment 
programs often have recourse to consultant contracts, particularly 
in the process of data collection and analysis* 

This stark suminary gives little sense of the actual nature of 
the various provisions for local progtamming and for Involvement of 
participants from outside the state and local education agencies. 
A few examples will suggest the variations in the substance and Ian-* 
guage of such provisions. 

- MARYLAND requires that "the State Board of Education 
and the State Superintendent of Schools, each Board 
of Education and every school system, and every 
school shall implement a program of educational 
accountability for the operation and management of 
the public schools..*" (Senate Bill 166 of 1972). 

Colorado goes farther — mandating not only local programs but also 
specifying the composition, tenure, and functions of state and local 
advisory Committees. 

- COLORADO* 8 Educational Accountability Act requires the 
creation of advisory committees at state and local level* 
At state level the Advisory Committee to the State Joard 
of Education must include 3 members from the House of 
Representatives, 2 from the Senate, 5 from present/past 
members of state boards of education, and 7 appointed by 
the Board-*~including 3 classroom teachers and 3 public 
school administrators (123«41*-3). The Board of Education 
of each school district must establish a local accounta*- 
bllity program and appoint an adviaoi^y accountablUty 
committee to pvike recommondat tout on that program. Com- 
mitteo mombera must Include at least one pnrenti one 
teachori one school admlnlitrator and one taxpayor from 
the district <123-A1-A). 
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California's statewide testing legislation does not specify the 
creation of special advisory commit teeSi but such groups are required 
uftder other statutes or orderss 

\ - CALIFORNIA has a system of advisory bodies created by 
\ statute ot executive order of the Governor* These 
^ Include an Educational Management and Evaluation Com-- 
mission (AB-2800) and a Joint Committee on Educational 
G04IS and Evaluation (appointed In 1970). The latter 
Committee Initiated a process of developing goals and 
objectives In 1972 by contacting large numbers of. 
citizens for their views. The Committee prepared a 
three-volume guide tc^ assist local groups In the pro- 
cess. 

Statutes In several other states Incorporate participation pro- 
visions other than the mandatory establishment of formal Advisory 
Committees. Thus^ Nebraska and Rhode Island require advisory con- 
sultation* 



- In RHODE ISLAND there is a requirement that the Board 
of Regents for Education "shall communicate with and 
seek the advice of those concerned with and affected 
by its determinations as a regular procedure in arrlv- 

1 ing at its conclusions and in setting its policy." 

(Ch.49, S.16(3), 1969). 

- NEBRASKA requires that the State Board of Education 
"provide for consultation with professional educators 
and lay leaders for the purpose of securing advice 
deemed necessary in the formulation of policies and 
in the effectual discharge of its duties.** (Bill 

No. 959| May 1969). 

In Virglniai the 1972 Act to Revise Standards of Quality and Specify 
Objectives for School Boards incorporates several provisions for 
the involvement of communityi prlncipalsi staffs and students in 
various aspects of school and classroom planning and management) 

- VIRGINIA requires that In each school "the principal 
shall involve the community and his staff In the 
preparation and Implcmontat ion of an annual school 
plan..." and that "the principal and his staff shall 
provldo for the cooporatlvo evaluation of the teachers 
and other omptuyoesi** The prescribed duties of 
teachers require chat thi^y "involve pupils in planning 
and conducting class act Ivit tos. < « " and ^^help eacti 
pupil to develop iUo ability to evaluate Ills own pro» 
greas arvl involve htm In the evaluation procapiii." 

(II 1972p H.l-a.hi K.2-eJ), 
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All three of the last mentioned states (Rhode Island, Nebraska, 
Vivglnia) have adopted goals for education that were generated in 
each case by a process involving participation by the State Educa- 
tion Agency, the legislature and a wide range of citizens throughout 
the state* 

The Trend to Provision for Broad Participation 

Beyond 5^tatewide testing/assessment legislation, there is evi- 
dence in other types of accountability legislation and in practice 
of a trend towards provision for participation of publics outside 
officially designated education agencies. 

Accountability statutes dealing with system management and 
evaluation of professional employees in some states require public 
participation. Thus, the statutes prescribing the requirements for 
non-mandatory PPBS in California (1967 and 1971) and Illinois (1972) 
entail public involvement. South Dakota's Teachers Professional 
Practices Act (1969, 1970, 1971) requires the creation of a Profes- 
sional Practices Commission with representation of teachers, board 
members and school administrators. 

Here, as in other dimensions of legislation examined, it is im- 
portant to keep in mind that the absence of statutory requirements 
does not mean absence of any designated feature in practice — partic- 
ularly where enactments give broad scope for interpretat ion« Imple- 
menting afiencies in several states are utilizing a range of practices 
to involvi^ communities and the general citizenry, particularly in 
generating goals and objectives. The "Our Schools" program in New 
Jordey, and the regional citizens' advisory councils, public hearings 
and conferences utilized in Illinois are examples.^ Local community 
participation is a aignif leant feature of PPBS variants which ompha- 
sice local goals and obJoctivoB--a f lacure that irorits monitoring 
as aymum managomont Btntuton and inactlcefi proliferate. 

Now Huxico providoH an iniiir&iil Ing vxampU of it«ituU)ry fipuct** 
tlt^niion of ( ItlKi^n pari tclpal luuf Ktati^wli)^ t4fi«(ln8 or «i'hm>) 
tivahmtlon \u aoi mamlMiut) In N^w H^xiio»"'{irol)ably l)i)rauiiM "inibllr 
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school officials have indicated that there are no funds to conduct 
a comprehensive evaluation of our public schools," However, a 1971 
Senate Memorial gives detailed guidelines for broad participation 
in District Educational Evaluation Committees: 

• NEW Mexico's senate, considering ''that concerned 
citizens, parents and teachers in each community 
are the ones who can best analyze the performance, 
effectiveness and needs of local schoolsit*" requests 
the State Board of Education to organize district 
Evaluation Committees comprising 10 members-- no more 
than 2 teachers and the remainder parents or con- 
cerned citizens residing in the district who do not 
hold positions of responsibility in the public school 
system. The duties of the Committees include orga- 
nizing public meetings on school Issues, broad con- 
sultations on directions for education, preparation 
of objectives, and recommendations for implementation. 
(Senate Memorial, 1971). 

The Memorial does not have the force of a statutory mandate, but sev- 
eral school districts in New Mexico have already begun local evalua- 
tion of the type described, 

The accountability statutes for Connecticut include one for 
experimental programs of innovation* The statute requires community 
participation: 

- CONNECTICUT requires school districts seeking to 
develop innovative educational programs to submit 
plans incorporating several features, one of which 
is provision for "direct participation by members 
of communities and students to be served by such 
experimental educational projects, in planning, 
policy-making and service functions affecting such 
projects." (Public Act No. A30, June 1971). 

Finally, it is notei that federal requirements and guidelines 
for ESEA programs Incorpoiate provisions for community participation 
that have stimuliced or supported new patterns of involvement in many 
states. Under Title I of ESEA, for example, USOE guidelines require 
ostab] Uhri^nt of Parent Advisory Councils for compensatory education 
programs* Their of feet ivimoBs varies from negligible, through 
nrutnil, to highly org^nUod and InfiuenciaK In Nov Jorsciyi for 
exnmpUi thay havo been termed "very active'' And a "ipoctacuUr 
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8ucce88|" with dramatic Improvement in reading skills of Title I 
pupils attributed to Increased Involvement of parents • Bovh national 
and state coalitions of **poverty parents^* have developed and there 
are statewide Parent Advisory Councils In at least ten states (circa 
March 1973). The USOE guidelines for councils confer no special 
power^ but parents define meaningful Involvement as ''the planning^ 
development! operation and evaluation of programs which affect their 
children."'' 

Thus, the patterns of involvement identifiable in statewide 
aeeeament statutes reflect and support a trend towards legitimating 
wide participation in accountability programs. 



C# Patterns and Trendsi 
Information Collection and Handling 

The Collection of Information 

Demands for accountability Incorporate pressures to generate 
more and '^better'' Information about schools and programs to guide 
decision-making. The legislation examined suggests variations In 
the extent to which states mandate* regulate, or permit Information 
collection, processing, Interpretation and dissemination. Evidence 
of Information assembled and processed In practice Is furnished In 
the ETS survey o\ state assessment programs and In documents from 
Individual states. 

Information collected may be grouped In three categories; 
assessments of cognitive development^ assessments of non-*cognltlve 
development, and '*related data'' on variables thought to bci associ- 
ated with scholastic achievement. 

Information on Cognitive Dcveloproent * The primary data assembled 
derive from testing procedures which furnish achiovomont scores pri- 
marily for basic skills and to aomo extent for content areas. Table 
2.7| abovc» noted the extent to whicli thOHO art^ Hpof^fied in I^rIh- 
latton. Table 2.9, below, dorlvotf from ikHcrlption« of Impl^montod 
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TARGET AREAS OF EVALUATION IN STATEWIDE TESTING /ASSESSMENT PROGRAMS 

Thirteen StatM - Circa 1971- 1973 
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X 
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X 
X 
X 
X 
X 
X 
X 
X 
X 



13 



X 
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Coi^itlve Domain 



Connecticut 

Arizona 

Maryland 

Rhode Island 

Californfa 

Maiiachusettt 

Florida 

Colorado 

Wi>con$ln 

Virflflnia 

Michigan 

Nebraska 

Penntylvania 



X 
X 
X 
X 
X 
X 



X 
X 
X 
X 



X 
X 



Non -cognitive 



Soiif ct 0«riv«d from « turvty o1 «iitsirti«nl proeramt by thf Education Tttting Sir^ct tnd otMf i. ^97 1. 
iuppltnitnttd by updattd mform«tion (or ir>divi(luat lUItt 



statewide assessment programsi confirms a heavy emphasla on cognitive 
learning and within that domain upon reading and computational (some-* 
times mathematics) skills. 
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Information on Non-^cof^nltive Development . Most of the statutes 
examined emphasize co*gnltlve sklllsi but many are capable of broader 
Interpretation* In practice} some states also collect Information 
based on tests and Inventories of non-*cognltlve development. Six 
states from the legislation sub-set do so. They suggest that Initial 
emphasis in non-cognitive assessment is upon self«-concept and upon 
attitudes to school and to scholastic achievement (see Table 2.9). 
Assessment programs in Pennsylvania, Colorado and Virginia may be 
precursors of a trend towards measurement beyond these areass 

- PENNSYLVANIA. The Educational Quality Assessment 
Program in Pennsylvania (initiated in 1963, with 
first statewide testing in 1970) Includes evaluation 
of self-esteem, understanding others, attitude to 
schooling, creativity, appreciation of human accom- 
plishments, and readiness for change. 

- COLORADO. The Colorado Statewide Learner Needs 
Assessment Program Includes evaluation of attitudes 
to school and to citizenship, self concept, and 
personal values in (current learning in) social 
sciences. 

VIRGINIA. The Virginia Educational Needs Assessment 
Study (involving a 10% sample of students in grades 
4, 7, and 11) Included evaluation of the affective 
domain in Phase I (attitudes. Interests, competencies 
in school and classroom settings, feelings of worth 
in Interpersonal relationships) and focused on the 
psycho-motor domain in Phase II (1971-72). 

There is a trend to Increased Interest in assessing non- 
cognitive areas. Implemented assessments are not widespread, but 
the reason n^ay be 'less that states or local districts are un- 
interested in testing non-cognitive dimensions than the present 
underdeveloped state of the art of measurement. Experience in 
Michigan suggests Instrumentation difficulties! 

- MICHIGAN'S Educational Assessment Program (first 
assessment 1969, fifth scheduled for Fall 1973) 
required measurement of self-concept and attitudes 
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TABLE 2.10 



INFORMATION "RELATED" TO STUDENT ACHIEVEMENT 
COLLECTED UNDER STATE TESTING / ASSESSMENT PROGRAMS 







Practice* 


State 


Legislation 


Input 
Variables 


Process 
Variables 


California 


M&EP 


X 




Michigan 


M& EP 


X 




Rhode Island 


IP 


X 




Virginia 


1 ID 

L !^.... 


X 




ArUona 


n 


X 


X 


Pennsylvania 


n 


X 


X 


Colorado 


n 


X 




Florida 


n 


X 




Wisconsin 


n 


X 




Connecticut 


n 


<■ 




Massachusetts 


n 


+ 




Nebraska 


n 


+ 




Maryland 


n 







Notes: M - mandated X ^ several variablei 

EP « explfcitly permitted * ^ very few vartabtes 

IP = irtiplicitiv permitted . n =» not mentioned 

' " accurate withtn the limits of available documentation 

Sourcei Derived from InforiYiation In statutes recorded by the Cooperative Accountability Project, 1972 - 73, 
from an ETS survey of state assessment progrems, 1d71« and from supplementary information for 
some individual states. 
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to school and to scholastic achievement in the 
first two assessments. These elements were ex** 
eluded from ensuing programs because the testing 
instrument was Judged defective • A new instrument 
will be available for the Fifth Assessment, but 
testing of attitudes will be optional • 

Information on Variables Related to Achievement i In order to 
"account" for the quality of student performance, many states seek 
to collect information on variables thought to be related to schol^ 
astlc achievement. Table 2.10 indicates the extent to which collection 
of "related data" is specified or permitted by statute and the nature 
of "related da£a" collected in practice* Several patterns of legis- 
lation are apparent. 

Sometimes collection of related data is mandated by statute. 
For example; 

- California's Education Code requires reports on 
results of testing programs to include "an analysis 
of the operational factors that appear to have a 
significant relationship to or bearing on the results" 
Including "but not limited to" demographic, financial, 
pupil and parent, and instructional and staff char- 
acteristics, and specially funded programs. (Ed* 
Code Section 12848 as amended by AB 665, 1972. The 
1969 formulation of the Section was more specifically 
prescriptive.) 

Sometimes the statute does not specify the nature of related 
Information to be assembled but explicitly permits Its col lectio?.. 
For example; 



- MICHIGAN. The 1970 Michigan statute explicitly permits 
collection and utilization of other relevant information 
essential to the mandated assessment program. In prac- 
tice, this allows assembly of information on school 
resources (human, financial, programmatic, facilities) , 
on socio-economic status (based on anonjnnous responses 
of students)^ and on district drop-out rates (derived 
from annual Department of Education studies). 



In fact, socio-economic status data have not been collected in 
Michigan's assessment program since 1971 and will not be collected 
in the Fifth Assessment scheduled for Fall 1973. 
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Sometimes the accruntability legislation neither permits nor pre- 
cludes the collection of related data, but in practice there is 
broad collection of non-test-score data* Arizona and Pennsylvania 
furnish examples, Arizona illustrates the case where the mandated 
focus of assessment appears to be narrowly circumscribed, but where 
in practice there Is a major effort to achieve some depth of informa- 
tion related to performance in the prescribed area. 

- ARIZONA, The 1969 Arizona statute mandating an 
annual standardized reading test for grade three 
pupils does not mention related data. However, the 
program implemented in early 1971 in response to 

the mandate involved collecting data on 21 variables, 
with 3-10 assessment levels for each variable. 
Categories of information pertain to: students 
(5 variables, including ethnic information, language 
spoken at home, socio-economic status); instructional 
programs (3 variabier?, Including the nature of the 
primary mode of teaching); teachers (3 variables on 
experience and formal education); classroom climate; 
school buildings, facilities, and services. Informa- 
tion collected in the initial run of the program and 
found not to be related to mean achievement differ- 
ences was to be removed from future annual testing 
data categories. 

Pennsylvania's Educational Quality Assessment Program (EQA) 
derives its mandate from a 1963 statute vrhich does not specify col- 
lection of "related data." EQA activities, initiated in 1967 to 
carry out statutory directives, incorporate major data collection 
and analysis of "educatioYial correlates." 

- PENNSYLVANIA. EQA Phase II involved collecting data 
from students (on background, community, availability 
of school resources), from teachers (responding to a 
76~item questionnaire designed to measure areas 
including job satisfaction, career aspirations and 
innovat iveness in the classroom), and from school 
administrators (on school programs and community). 
Supplemental data were obtained from the Department 
of Education (e.g., teacher sex, age, experience, 
education, salary; district financial data). 

From these raw data analysts derived indicators of: 
(a) school and community characteristics (6 variables 
on program resources, 3 on financial resources, 6 on 
demographic characteristics); (b) instructional staff 
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characteristics (6 on teacher background, 5 demo- 
graphic, 6 on attitudes); (c) student indices (6 
on background, 5 attitudinal, 4 demographic) • 

The Pennsylvania program is not typical, either in the amount 
or the kinds of "related data" collected, of state assessment pro- 
grams generally or of the sub-set analyzed. Some of the states 
examined collect "related data" on many variables (e.g., Colorado, 
California), some Qn very few (e.g., Nebraska and Connecticut). 
The most commonly collected data pertain to student background and 
to schools • Within those categories the information sought is 
largely demographic (sex and age of students; name and size of 
schools) and on socio-economic background. Six of the states col- 
lect data on teachers, with eonphasis on demographic variables (sex, 
age, formal education, salaries) rather than behavioral/performance 
indicators. Information on program cost — a fundamental datum for 
purposes of accountability analysis — was specified in descriptions 
of only four programs in the set. Thi^re is a dearth of information 
on process variables, instructional styles and programs. 

General findings on information collected in statewide testing/ 
assessment programs generated under accountability legislation are 
thus: (1) All states oolteot ^^autput^^ information in the form of 
test scores » Most emphasise limited dimensions of cognitive learn-- 
ing but there are indicators of a trend to broaden the range of 
learning assessed. (2) Nearly all states collect ^^related data" 
whichj in practice, are almost entirely "input^^ information empha- 
sizing student, teacher, and school demography and the socio- 
economic background of students. (3) Very few states collect 
^^process" information via indicators of variables in teaching/ 
learning styles and classroom behaviors and practices. 
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Analysts and Interpretation of Information 



What happens to test data and related Information once they 
have been collected? Who analyzes and interprets the data for whom? 
Here again, there are variations among states, 

- Prooessing external^ interpretation external n Both 
largely in the hands of professional testmakers/ 
evaluators outside state/local education agencies. 
Example: COLORADO Evaluation Project, May 1970 
pilot run--Instruments were developed and results 
processed and interpreted by Pacific Educational 
Evaluation Systems (PEES), California. 

- Prooessing external^ interpretation interncl/exterml. 
In this pattern interpretation is done with more or 
less collaboration between external agencies and state/ 
local education agencies. Examplot MICHIGAN Educa- 
tional Assessment — processing done by the Educational 
Testing Service, New Jersey; interpretation by Michigan 
SEA with the assistance of ETS.* 

- Prooessing extermly interpretation internal. 
Example: ARIZONA Statewide Reading Testing Program — 
Measurement Research Center (MRC), Iowa City, scored 
tests and taped raw and derived scores and related 
data; the tapes were interpreted by Arizona SEA. 

- Prooessing and interpretation internal. Example: 
CALIFORNIA Statewide Testing Program^— Local diotricts 
score, summarize and report testing results on SEA- 
designed forms. The SEA is responsible for statewide 
analysis and interpretation of program data. 

Clearly, there is considerable leeway in implementing legislated 
testing/assessment programs. This is reflected in variations in the 
extent to which data collection, processing, and interpretation are 
contracted out, performed by the state education agency, or diffused 
among local agencies. Few statutes specifically mention such arrange- 
ments, but practice may often Involve recourse to specialized consult-- 
ing services. The interpretation function Is particularly sensitive.*' 



Michigan changed its contractor to MRC in 1973. 
See discussion below, page 107 ff . 
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It relates processing and analysis, where the statutes are not pre- 
scriptive, to dissemination, where they are. 

Dissemination of Information 

Who gets the information in what form? Here again there are 
variations among states. Table 2,11 summarizes the specific statutory 
provisions on these matters, 

A common pattern of formal reporting is for the State Depart- 
ment of Education to transmit the results of testing/assessment to 
the State Board of Education, The Board then makes recommendations 
to the Legislature* There are varying provisions with regard to dis- 
semination of results to local school districts, individual schools, 
and "the public," Local districts usually must report local data to 
their state boards or departments. And state agencies typically 
report data and analysis to local district administrators* Several 
states distribute copies of formal reports to teacher organizations, 
and many utilize newspapers or other communications media to release 
program results to "the public*" 

The nature of information reported to any group or individual 
is a matter of some sensitivity. In some states, the forms in which 
information may be reported are neither prescribed nor proscribed by 
law; in others they are, Florida and California offer examples of 
specific statutory provisions: 

- In FLORIDA, the 1971 Educational Accountability Act 
requires the commissioner of education to make- an 
annual public report of assessment results including, 
"but not limited to" results by grade and subject area 
for each school district and the state, together with 
analysis and recommendations concerning the costs and 
differential effectiveness of instructional programs. 
By 1973-74 school boards in each district must make 
annual public reports of this type, 

- CALIFORNIA'S Education Code includes prolific provi- 
sions for testing, including the nature of required 
reporting. District-wide results of mandatory testing, 
but not the score or relative position of individual 
pupils, must be reported at least annually to the 
governing board of each district. The district board 
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TABLE 2.11 



PATTERNS IN STATEWIDE TESTING / ASSESSMENT LEGISLATION 
REPORTING PROVISIONS 
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Rhode Island, 1963 
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Virginia, 1972 












Wisconsin, 1971 












Frequency 


7 


8 


4* 

[13] 


7 


4 



Mot«: •State Board) / Department of Education ©enefaUy havd admmiitrative control of the 
implementing of statutes (See Table Vlll). 

Source: Texts of legislation reported throus^ the Cooperative Accountabifity Project in 1972 and 
April, 1973. 
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must report school-by-school results of tests to 
the Department of Education, (AB,665, Ed. Code 
12826, amended). The Department of Education must 
make annual reports to the Legislature, to the 
State Board of Education and to each school district 
in the state, giving district-by-district analysis 
of testing results, including analysis of "opera- 
tional factors that appear to have significant rela- 
tionship to or bearing on the results." (Ed. Code 
12848 as amended by AB.665, 1971). 

In states where there is no detailed statutory prescription 
for reporting, in practice the implementing agencies establish pro- 
cedures and guidelines. Massachusetts and Rhode Island provide 
illustrations: 

- MASSACHUSETTS. Under the Massachusetts Design for 
Assessment (initiated in 1970 with first assessment 
in 1971), the State Education Agency has distributed 
state summaries of program results to the State Board 
of Education and to school districts* This statewide 
reporting did not identify individual districts or 
schools. Mechanisms were provided to allow individual 
districts, schools, and teachers to examine results 

on an individualized basis. 

- RHODE ISLAND. Reports from the Rhode Island Statewide 
Testing Program are distributed by the State Education 
Agency to school districts, schools and teacher organi- 
zations. Further dissemination is determined locally. 
While there is a* general commitment to public reporting 
of results, the form and manner in which test scores 
should be reported is under examination by a study. com- 
mission appointed by the State Board of Regents. 

Where rather small samples are used in assessment programs, 
there may be deliberate restraint in reporting detailed results. 
Thus! 

- The VIRGINIA Educational Needs Assessment Study in 
1970 dealt with only about 10% of the school popula- 
tion in six sampled geographic areas. No reports 
were prepared for individual schools. The only 
information disseminated was analyzed by geographic 
areas and released to the press, so that broad pat- 
terns of performance and needs in the state were 
made public* 
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TABLE 2,12 

TRENDS IN ACCOUNTABILITY REQUIREMENTS INCORPORATING 
MOUNTING PRESSURES FOR EVALUATION DATA 



Trends 

- 


Indicators 


Mandatory statewide test^n^ / assessment 


- Legistation in 13 states; (ikety in others 

Most states have some kind of assessment program, 
including evaluation data 


Collection of data "related" to student 
achievement 


- Legistated requirement in 2 states; practised in 
many states; increasingly associated with account- 
ability programs 


PPBS (planning-programming-budgeting 
system) or PP6ES (. . . -evaluating system) 


- Legislation: Statutes in at least 8 states 
(at feasts acts since 1970): 
California (1967, '71) Indiana (1971) 
Florida (1968, '70, 73) Colorado (1971) 
Alaska (1970) Illinois (1972) 
Hawaii (1970) Ohio (1972) 


MIS/UAS (Management information / 
Uniform accounting system) 


- Legislation in at least 5 states since 1967: 

New Mexico (1967) Nebraska (1971) 
Rhode Island (1969) Arizona (1972) 
Florida (1970) 


Mandatory evaluation of performance 
of professional employees 


- Legislation in at least 8 states 
(at least 7 acts since 1971): 
Washington (1969) New Jersey (1971) 
Florida (1969) Oregon (1971) 
S. Dakota (1969, 71) Connecticut (1972) 
California (1969, 71) Virginia (1972) 


Competency-based teacher certification 


- Florida, New York and Washington have taken a 
leadership role in state policy on this 

- Many other states are supportive, including Utah, 
Texas, Kansas, Nebraska, California, Oregon, 
New Hampshire, New Jersey, Minnesota 


Comprehensive planning in State 
Departments of Education 


— 34 states reported planning units in 1971 survey 
by Minnesota State Department of Education 

Estabtished 1966 - 2 1969 - 6 
1967 -3 1970-20 
1968-2 
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Trend Towards Increased Pressure for Data Collection 



and Handling 

We have been considering data generated by statewide testing 
and assessment programs. If we add to these the Information required 
or permitted under other types of legislation and practice, further 
aspects of the growth of education data banks In the states are 
apparent . 

Table 2.12 suggests evidence of trends towards requiring evalua- 
tion of the performance of professional employees, competency-based 
teacher certification, program-budgeting and management Information 
systems, and comprehensive planning. Thus we note that the data 
collected on student performance feeds only one of many streams flow-- 
ing into rapidly growing reservoirs of information about education 
in the states. 



D . Patterns In Legislation; 
Uses of Evaluation Information 

What kinds of decisions are to be made on the basis of the evalu- 
ations generated under statewide testing/assessment legislation? Some 
answers to that question are to be found both In the general expres- 
sions of legislative Intent and In the sections of statutes that specify 
reporting requirements. 

Table 2 . 13 summarizes the uses of evaluation Information specified 
In the legislation. Those uses range from application to decisions 
about particular programs to decisions about the general quality of 
public education statewide. Thus in ARIZONA test information is to be 
used to "enhance the quality of the reading program in the public 
schools," whereas in COLORADO it will contribute to decisions designed 
*'tc help the public schools of Colorado to achieve quality and to 
expand life opportunities and options of the students of the state." 

There is specific reference in five statutes to the use of evalu- 
ation for diagnosing individual learning problems and for improving 
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PATTERNS IN STATEWIDE TESTING / ASSESSMENT LEGISLATION 
SPECIFIED USES OF EVALUATION INFORMATION IN DECISION-MAKING 
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Nebraska, 1969 
Wisconsin, 1971 
Massachusetts, 1965 
Florida, 1971 
Rhode Island, 1969 
Connecticut, 1971 
Maryland, 1972 ^ 
Pennsylvania, 1963 
California, 1972 
Colorado, 1971 
Virginia. 1972 
Michigan. 1970 
Arixooa, 1969+ 1972 



X 
X 
X 
X 
X 



X 
X 
X 
X 

X 
X 
X 



X 
X 
X 
X 
X 
X 
X 
X 
X 
X 
X 



X 
X 



X X 
X 



Note: •Othef uses $pecifi«d in itatutw incfud* "rweafoh" in California, snd "accreditation" in Florida. 



Sourc«: Derived from ejominatton of itatutet recorded in the Cooperativa Accountability Project lo April 1973. 
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individual performance. Lack of such reference in other statutes 
does not imply that concern for individual students is absent. How- 
ever, there is an important shift in focus in contemporary legisla^ 
tion which broadens the scope of decision-rmking to he served by 
prescribed evaluation. Formerly the most frequently stated use of 
test data was to aid local decisions about the guidance of individ-- 
ual students. That use remains. But under contemporary legisla-- 
ticn, the primary emphasis is upon aiding state-level decisions 
about ''programs and ''the system. 

Other legislated uses of evaluation include planning (Rhode 
Island), research (California), accreditation of schools (Florida), 
cost-performance analysis (Colorado and Florida), and allocation of 
resources (California and Michigan ). 

Cost-Performance Analysis * COLORADO and FLORIDA explicitly 
require analysis of "the costs and differential effectiveness of 
instructional programs." Maryland requires provision of information 
for such analysis. This type of requirement, in which data from 
student testing are related to cost and expenditure data, implies 
the existence or development of a unified budgetary system; in fact, 
the requirement is more common in PPBS-r elated legislation. Thus, 
it is not surprising that Colorado is developing a program planning, 
budgeting, and evaluation j-iystem (PPBES) for public schools under a 
1971 statute (Senate Bill No. 42). And the Florida Department of 
Education is seeking to meet the stipulation for cost-performance 
analysis contained in the Accountability Act of 1971 by developing 
a n^odel for a PPBS at the local district level. 

Resource Allocation and the Legislature . The CALIFORNIA Educa- 
tion Code, as amended in 1971, specifies that evaluation information 
is to be provided as a basis for allocating educational resources. 
Thus, the results of first grade entry testing are to be used to 
determine each school district's quota of specialist reading teachers. 

In Michigan this specified use of test data is in the context of 
the compensatory aid program. 
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The formula for allocating specialists by those results is detailed 
(Ed.Code 5779 and 5782, 1971), This use of testing reflects a more 
general specification in the Code requiring that the statewide test- 
ing program provide evaluative infornvation so that> inter alia^ "the 
Legislature and individual school districts may allocate educational 
resources in a manner to assure the maximum educatioiial opportunity 
for all pupils." (EdXode 12821, amended 1971), California and 
Michigan have the only statutes examined which explicitly refer to 
the use of testing information as a basis for resource allocation— 
although this use can readily be implied for other states. Statutes 
in five states specify that the implementing ai^,ency is to make recom- 
mendations to the Legislature in its reporting, and remaining statutes 
do not preclude this. On the basis of evaluation information, there- 
fore, the agency can make recommendations to guide policy-uial.ing 
(whether by means of legislative or other instruments) affecting the 
future of education in the state. 

Information about Implemented state assessment programs confirms 
an increasing emphasis on using program results (mainly test data) 
for state-level decisions about resource allocation and for planning 
purposes • 

Proscribed Uses of Test Information 

CALIFORNIA, which has the most copious legislation on testing, 
explicitly proscribes certain uses of evaluation information* The 
relevant provisions in the Education Code affect the use of test 
scores in relation to high school graduation, promotion to different 
grade levels, first grade placement, and cumulative school records 
in grades one to three: 

- Testing legislation is not to be construed to mean 



"that graduation from a high school or promotion to 
another grade level is in any way dependent upon 
successful performance on any test administered as 
part of the testing program." (Art.l, 12830, 1969). 

- Test scores or results are not to be employed to 
rank school districts in any publication other than 
necessary for administering the statute (Art. 3, 5779, 
1965— Mlller-Unruh). 




- "Scores for individual pupils on the first grade 
entry level test shall not be used by school 
districts or teachers for individual diagnosis of 
placement or as a basis for any other decisions 
which would affect the pupil's elementary school 
experience* Scores from this test shall not in 
any manner be included on the pupil's cumulative 
school record." (Ed. Code, 5779.2, 1971). 

- "The State Board of Education shall determine which, 
if any, of the scores attained by pupils on the tests 
administered in grades 2 and 3 may be recorded on 

the pupil's cumulative school record." (Ed. Code, 5779.2, 
1971)- 

The prescriptions and proscriptions for the use of evaluation 
data are important. As we shall see in Part III, several problems 
are likely to be encountered in the sensitive area of utilizing 
evaluation information in decision-^making. 



ERLC 



lOA 



Recapitulation of Patterns in Statewide Testing LeQiBlation 



Purposes and Intentions 

- The primary intent of recent legislation is to assess the per'- 
formanoe outcomes of schooling statewide in relation to the 
public resources going into education. The information gener-^ 
ated by evaluation procedures is intended to serve the purpose 
of informing the ^^publio" and/or its representatives of what 
has been achieved in the my of learning in return for expendi- 
tures of their tax dollars. It is in this sense that the statutes 
are concerned with accounting and accountability. The information 
is also intended to feed into a management information pool as 

a resource for decision-making affecting the future of education. 

Goals and Objectives 

- Two principal patterns are distinguishable in testing statutes. 
Pattern A statutes require evaluation of the achievement of 
objeotives--specified in varying degrees — without referring to 
processes or mechanisms for determining goals and objectives. 
Pattern B statutes specify or imply a process wherein goals and 
objectives are examined as an integral part of ongoing actixyities 
of planning and evaluation — of which testing is one component. 

Performance Objectives and Testing Priorities 

- Both statutory prescriptions and available information on imple- 
mented testing programs indicate strongly that ^^quatity of edu- 
cation^' is assessed primarily by measuring performance in ''basic** 
cognitive skills. 

A dministration and Participation 

- There is clear emphasis in all the statutes on the state educa- 
tion agency as the locus of control and responsibility for imple-- 
menting testing /assessment legislation. Patterns of involvement 
of people outside the state agency^ however^ are identifiable in 
several statutes. These both reflect and support a trend towards 
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tegittmating and stvmlating wide participation in aooounta-^ 
bility programs* 

Information Collection and Handling 

- Information oolleoted under testing /assessment legislation may 
be considered in three categories: assessments of cognitive 
development (primarily in basic skills^ to some extent in con- 
tent areas) 3 assessments of non-^cognitive development (not 
mandatory^ but assembled in some states )s and '^related data. 
Collection of data '^related'' to achievement is not mandatory 
in many states^ but it is common in practice. The information 
is almost entirely on ^Knput^* rather than ''process" variables; 
very few states collect information pertaining to teaching/ 
learning styles and classroom behaviors and practices., 

- Processing . While few statutes specify suoh arrangements^ in 
practice many administering agencies have recourse to special- 
ized consulting services for the design of instruments^ data 
collection and handling. 

- Reporting * A common pattern for formal reporting of testing/ 
assessment information is for the State Department of Education • 
to transmit the results of evaluation to the State Board of 
Education which makes recommendations to the Legislature with 
the general intent of improving the quality of education (as 
measured by performance levels on tests). Below state levels 
there are varying provisions for dissemination of results to 
local education districts^ individual schools^ and ''the public.^' 

- Trend to increasing data hanks * There is evidence in the states 
of trends towards requiring evaluation of the performance' of 
professional employees^ competency-based teacher certification^ 
planning --programming "budge ting and management information sys- 
temss and comprehensive planning. Thus^ data collected on 
student performance under contemporary legislation feeds only 
one of many streams flowing into rapidly growing reservoirs of 
information in the states — information to be interpreted and 
used in decision-making affecting the future of education and 
its publics. 



- Uses of infommtion . Formerly^ the most frequent use of test 
data was to aid local decisions about student guidance* That 
use 'remains. However^ the primary emphasis in contemporary 
legislation is upon aiding state-^level decisions about ^^needs^** 
'^deficiencies^ " and the general improvement of the ^'quality of 
education^' stateiHde. This represents a significant broadening 
of the scope of decisionmaking to be served by test data* 



PART THREE 



BEYOND THE LE G ISIATION; 
F ROM EVALUATION TO D ECISION-MAKING 

An Examination of Some Problems 

Our purpose In this Part of the report is to discuss some of 
the problems involved in using test information in decisions of 
edui^.atlonal policy. There is no attempt here to formulate these 
problems in a way amenable to policy analysis. The discussion does 
not seek to judge which of them can or should be the target of what 
kinds of policies at what levels of government* They are problems 
of consequence, however. They arise in practice and they can be 
identified and anticipated in the course of policy planning. 

The discussion falls into three sections. Section A suggests 
some problems of State Education Agencies (SEA's) arising from 
mounting demands for data collection and handling. Section B con- 
siders problems associated with fears of various groups about the 
interpretation and use of evaluation data. Section C abstracts 
from problems of tests, testers and testing, to consider whether 
the Information engendered under statewide testing legislation is 
likely to be adequate for Intended uses in decision-making. 



A. Data Management and SEA Capabilities 

Pressures are mounting for more and more Information for deci- 
sion-making purposes. The demand stems not only from statewide 
testing, but from PPBS and related efficiency-accounting systems, 
teacher evaluation, performance contracting, and from the trend 
towards comprehensive planning for education at the state level. 
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These pressures are likely to place strains upon the already taxed 
capabilities of many state education agencies. Moreover, these data 
management demands Increase the need to develop and strengthen pro- 
cesses In which continuing attention Is paid to the purposes. Inter- 
pretation, usesj accessibility and dissemination of Information. 

Consider In brief: 

- Outlays for complex computer facilities and for tech- 
nically trained personnel contribute to already 
increasing demnds upon scarce resources available 
for education In the states* 

- There Is evidence of a propensity to purchase tech-* 
nlcal "expertise" not available within the SEA. This 
can be a logical and effective solution, but it requires 
careful examination — especially when It Involves hiring 
outside experts to do goal analysis, needs assessments, 
and Interpretation of sensitive data. 

- Some sea's are experiencing difficulties In recruiting 
and retaining skilled personnel to support their 
expanding functions In management, planning, and tech- 
nical assistance. This Is partly a problem of salary 
structures and the scarcity and vicissitudes of funding. 
However, It Is likely that It will be easier to Identify 
technical needs and to recruit technical experts when 
hiring new personnel than It will be to Identify and 
recruit or train people who can deal with questions of 
purposes, policy alternatives and consequences, and the 
participation of multiple publics. 

- Behaviors, attitudes, and views of technical experts 
may run counter to concerns for grassroots participa- 
tion. The possible conflict of views should be examined. 
There may be a propensity to Interpret "participation'^ 

in the policy-planning process as handling some kind 
of "dissemination" activity — unless there is deliberate 
intention, action, and educating to the contrary. Tech- 
nical assistance as a participatory learning activity 
for all involved is scarcely a popular interpretation 
of the function in bureaucracies. 

Here, as elsewhere, the intent is not to suggest gloomy scenarios 
it is rather to point to some of the challenges presented by the cur- 
rent wave of statewide testing and other accountability legislation. 
The administration and interpretation of that legislation falls 
heavily to state education agencies. The implementation of account- 
ability programs will require not Just more funds and more staff, but 
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a re-examination of requisite staff competencies and training, 
B. Evaluation and Decisions; Some Fears and Pitfalls 



Many of the problems meriting careful examination by those 
designing and implementing accountability programs pertain to the 
sensitivity of the evaluation information and the fears of various 
groups that it may be interpreted and used in ways adversely affect- 
ing their interests. 

The Sensitivity of Evaluation Data 

On the one hand, there is a general societal concern for some 
• transparency in the decision-making process and for public access 
to information on which policy Judgments are based. On the other 
hand, there Is controversy over human privacy vs. mass data banks 
for planning and research, and over the interpretation and misin- 
terpretation of test data. 

There is evidence of mounting concern for the confidentiality 
and legitimacy of certain types of information supplied by students, 
teachers, and others. When information is required about personal 
attitudes, socio-economic background, ethnic origin, and similar 
matters, questions of invasion of privacy and suggestions of psycho- 
logical damage can be and have been raised. And, of course, pos- 
sibilities for invalidating the information by falsifying responses 
must be considered. 

These are not just matters of speculation* There is sufficient 
evidence associated with testing programs examined to verify that 
there are real political, ethical, and technical-analytic problems 
related to the sensitivity of Information and its interpretation. 
In some cases they may seem intractable. Credibility of public edu- 
cation agencies and evaluation experts may be in question. For 
example, the Educational Testing Service reports in relation to the 
Michigan Educational Assessment Program that: 
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- In MICHIGAN some schools refused to return student 
answer sheets provided In the Assessment Program on 
the grounds that responses might be used to penalize 
them because of background or to malign the reputa- 
tion of their ethnic groups — even though the data 
were gathered in a manner that guaranteed anonymity 
of respondents, and even given the announced intent 
to use the information only for estimating the over- 
all impact of educational programs on each of several 
target populations. 

Further: 

- In MICHIGAN legislators wanted comparative data on 
individual districts from the 1970-71 Assessment 
Program. Many school officials, however, object 
to public release of data for individual schools 

and districts — despite the fact that the Superintendent 
has ruled they be released. 

Test-related data are sensitive not only because they may involve 
information some people consider private. They are sensitive also 
because they are susceptible to misinterpretation. Possibilities for 
drawing invalid inferences from test scores and for making invidious 
comparisons are real. Hence some states prohibit the release of test 
scores which would identify individuals or schoolSj or even rank 
districts . 

Consider the recent debate over dissemination of standardized 
test scores in New York City and in New Jersey. 

- NEW YORK CITY again published results of city-wide 
reading tests administered to students in 1972, listing 
performance by school and grade level and giving, for 
each grade level, a report on the percentage of students 
falling behind the national norm. Front-page coverage 
in The New York Times and prominent space in some other 
newspapers conveyed that 66.3% of elementary school 
pupils and 71.3% junior high and intermediate school 
pupils were reading below grade level. The implication 
was that more than two-thirds of the students were 
"failing" and were "sub-normal." 

Albert Shanker (President of the NYSUT, the largest teacher union in 
the country), commenting in The New York Times on this publicity, 
notes that half the students must be below a norm no matter how well 
they read. Failure to understand this "has resulted in recent years, 
in odious comparison between schools, and militant attacks upon 
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teachers and supervisors by parents of children In schools with low 

reading scores,** In many cases fewer than half the children tested 

In a given grade, school or district In New York City In 1972 were 

there to be teste<t In 1973, Given this high mobility of pupils, test 

scores cannot provide a valid measure of whether students are doing 

better or worse from year to year on a school-by-school, dlstrlct- 

by-dlstrlct or even clty-wlde basis. Test scores may reflect prior 

schooling In other schools, districts, and even In other cities or 
9 

states. 

In such circumstances, it Is not surprising to find attempts to 
block the publication of test scores. Thus: 

- In NEW JERSEY the New Jersey Education Association, 
sought to block the dissemination of standardized test 
scores In the state. They argued that mere publication 
of standardized test scores might hinder rather than 
contribute to public understanding. Unless there Is 
sophisticated Interpretation, scores become more of a 
weapon In the political process than an aid In Judging 
the educational process. 

The National Assessment of Educational Progress has come under similar 
attack for reporting test results without Interpretation. Thus: 

- In late 1972 representatives from 23 large city school 
districts accused NAEP of •'pot'entlally dangerous" 
methods of reporting assessment results* The Board 

of Directors of the Council of Grt^t City Schools 
pointed to two factors which could lead to **caprlclous 
and uni^rranted Interpretation of testing results" — 
the policy of NAEP to report uninterpreted data, and 
the use of "subcategories" which may be "too small In.^ 
size to be representative of the sampled population." 

Two Senate resolutions In Hawaii in 1972 take account of the 
controversy. They reflect both a response to demands for transparency 
in sharing Information and a concern for careful Interpretation to 
accompany the sharing or Information. Thus: 

- In HAWAII one Senate resolution In 1972 requires 
revised methods of reporting results of tests and 
says that "public concern has created the need for 
widespread comprehensive reporting of the progress 
and problems of education In the state." A concur- 
rent resolution requires that routine publication 
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of results be supported by interpretations* explana- 
tions and analysis to place test results In perspec- 
tive and prevent misunderstandings from arising as 
to their meaning. 

Mislabeling and Discrimination: Fears for Students 

Test bias and the use of tests for tracking and labeling stu- 
dents have been the subject of Increasing public outcry In several 
states. Minority groups In particular have suffered from mlsclas- 
slflcatlon and attendant disadvantages associated with the use of 
culturally biased tests. The arbitrariness of the labeling deci- 
sions made on the basis of tests Is Illustrated In the case of one 
state In which It Is reported as having leen common practice to place 
In special education classes any child who scored 75 or below In a 
diagnostic test administered to all children, whereas a score of 76 
guaranteed a place In a regular class* 

Litigation has been Involved In some states. Thus: 

- In Boston, MASSACHUSETTS a suit was brought In 1970 
charging that the Boston school system and the Massa- 
chusetts State Department of Education^ through a 
faulty method of testing and classification, placed 
large numbers of children Into special classes for 
the mentally retarded. Damages were sought for each 
of seven children allegedly "irreparably harmed" by 
the classification system. ^ 

- in CALIFORNIA several court decisions and research 
studies blamed scores on group IQ tests for unfair 
placement of many black and Chicano students in special 
classes. 

The California State Legislature's concern over these alleged mis- 
classifications led to a law two years ago encouraging the transfer 
of many students out of special education classes after re-evaluation. 

In Colorado a current bill is seeking to Improve the quality 
of the tests used with various cultural groups: 

- In COLORADO, HB 1478 asks that tests used in the 
schools be free from cultural and linguistic bias 
or separately normed with reference to linguistic 
and cultural groups to which the child belongs. 
The State Department of Education is to determine 
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which tests are free of such bias. 



Progress In developing such sensitivity In testing has barely 
begun, however, and criticism of tests, testers and testing by 
minority groups Is Increasingly bitter* At a recent symposium on 
testing black students, one participant proposed that a ''survival 
quotient'' would be a better index of a black child's learning abil- 
ity than the traditional intelligence quotient. Another urged that 
the federal government regulate standardized tests as they do drugs 
to ensure that they would be ''safe and effective for their intended 
use." 

Minority groups are launching major efforts to heighten aware- 
ness of testing abuses and to develop instruments which will be more 
sensitive to cultural differences among students. The First National 
Conference on Testing in Education and Employment was held at Hampton 
Institute in April 1973. It attracted some 450 participants from 
education, labor, industry, government, and private and public com- 
munity organizations. They met to explore the "ways assessment 
Instruments have been used to exert a malevolent influence on the 
social mobility of minority children, youth, and adults, and also 
to pinpoint the ways in which testing serves as a prop for tliose 
supporting a racist and oppressive system of educational and employ- 
ment practices, "^^ A national project to curb bias in testing was 
launched with this conference. The project director, Noiman R. Dixon, 
seeks to have the project serve as a bridge between test makers and 
their critics. He said: 

The central overriding purpose is to improve the .quality 
of American life through making significant changes in 
standardized testing. ^5 

Not long after the Hampton conference. Professor Dixon was to 
comment on the decision of a U.S. court Judge in Georgia as "one of 
the crassest examples of the misuse of tests in the history of edu- 
cational testing." The opinion is noteworthy because it Involves a 
decision about school quality, based upon test scores, and requires 
a desegregation plan from a State Board of Regents to raise quality 
level • 

Er|c 111 



- GEORGIA, The case in Hunnlcut Surge . U.S. 
District Court Judge Wilbur D, Ovens has ruled that 
a black Georgia college is a ^'diploma mill" on the 
basis of the test scores of its students. He con- 
cluded that its academic inferiority is associated 
with its racial identity, and ordered a desegrega- 
tion plan that includes academic upgrading of the 
college, (Note: The judge considered that one of 
the two major tests Involved, the Georgia Rising 
Junior Test, did "not appear to be subject to pos- 
sible attacks of cultural bias,")l6 

Unfair Accountability: The Fears of Teachers 

The National Conference on Testing referred to above was held 
in a climate of mounting opposition to testing among educators. The 
National Association of Elementary School Principals has urged re- 
sistance to any use of standardized testing for purposes which can- 
not benefit the child. Some groups, including the Association of 
Black Psychologists and the National Education Association, have 
called for a moratorium on standardized testing* 

Teachers are among the most vocal and organizfid groups taking 
a stance in matters of accountability legislation and regulations. 
Fears of teachers that they will be unjustly held responsible for 
measured learning performance of students and that hard-won victories 
of status and tenure will be wrested from them have impeded the pas- 
sage of legislation in some states. Two pieces of litigation suggest 
the basis for teacher fears: 

- In IOWA, in the case of Norma Sheelhasse v, Woodburn 
Central Community School District, 1972 , a federal 
district Judge has ruled that a teacher cannot be 
dismissed solely on grounds that her students did 
not do well on tests (Iowa Tests of Educational De- 
velopment and Iowa Tests of Basic Skills) , The 
school district has appealed the decision. 

- In CALIFORNIA, the case of Peter Doe v. San Francisco 
Unified School District Involves a student who gradu- 
ated from high school but can read only at fifth grade 
level. The complaint contends that the plaintiff has 
been deprived of an education in the basic skills of 
reading and writing as a result of the acts and omis- 
sions of the defendants who are: the School District, 
its Board of Education and Superintendent of Schools; 
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the State Department of Education, its Board of 
Education, and the State Superintendent of Public 
Instruction; and one hundred (unnamed) defendants 
alleged to be the agents or employees of the public 
agencies • 

In a unique conference prior to liling of the Peter Doe suit a group 
of ,ibout sixty people (lawyers, educators, state and federal govern- 
ment representatives and school administrators) discussed the case 
and its legal and educational implications. The strongest case for 
the defense was marshalled by Judge Haskell Freedtnan who heavily 
emphasized the case for the teachers included among the defendants. 
They can, he contended, be held accountable neither legally nor edu- 
cationally for the poor learning outcomes of the Peter Does of our 
schools: 

- I accept the proposition that teachers should be 
educationally accountable, but add that to try to 
hold the teaching profession solely or substantially 
accountable under existing conditions in our public 
schools is injust, inequitable, and an attempt to 
use teachers as scapegoats for the failure of the 
educational process.l9 

We note that the Peter Doe complaint has been filed in Cali- 
fornia. Among the eight states now reporting legislation for the 
evaluation of professional employees,* California Is the only one 
that explicitly requires assessment of personnel competence in 
relation to standards of student progress: 

- CALIFORNIA. AB 293, enacted July 1971 (the "Stull" 
bill): The governing board of each school district 
shall develop and adopt specific evaluation and assess- 
ment guidelines which shall include but shall not 
necessarily be limited in content to the following 
elements: (a) the establishment of standards of 
expected student progress in each area of study and 

of techniques for the assessment of that progress, 
(b) assessment of certified personnel competence as 
it relates to the established standards... Each Board 
is required to "avail itself of the advice of the 
certificated instructional personnel from the district's 
organization of professional personnel." (Ed. Code 
13486-7). 



See Table 2.4, page 66. 
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And we note that California again leads the way In legislating the 
development of minimum standards for high school graduation: 

- CALIFORNIA. AB 655, 1972: The governing board of 
an7 school district maintaining a high school shall 
adopt minimum academic standards for graduation from 
the high schools within its school district... 

(Ed. Code 8574, amended 1972). 

Teacher unions have taken a strong stance on accountability 
legislation. Delegates to the 1972 rr^eting of the National Educa- 
tion Association called^ inter* alia^ for a national moratorium on 
standardized testing^ for legislation to outlaw educational voucher 
plans, and for an end to performance contracting in public schools. 
Early in 1973 the NEA issued a declaration on educational accounta- 
bility. The tone of commentary reporting that event suggests some 
of the strong feelings involved. The New York Teacher (official 
organ of the NYSUT) reported thus: 

- Acting In defense of victimized teachers tmd students, 
the 108-member NEA Board of Directors last: month 
unanimously approved a declaration on educational 
accountability. NEA president Catherine Barrett 
praised the Board action and noted that a recent NEA 
report reveals that ^'compulsion about accountability 
in education has reached crisis proportions in at 
least 30 states and is spreading fast to all 50." 

The Report on Education Research reported in this way: 

- NEA SEES EDUCATIONAL FASCISM IN ACCOUNTABILITY PUSH. 
The accountability movement is a "warped attempt" to 
apply corporate management system models to education, 
and it threatens students and teachers with "punitive, 
ill-conceived and probably inoperable" legislation 

and directives, NEA charged in a report to its members. 
It urged teachers to turn the trend around. Accounta- 
bility misapplied can lead to a "closed system — educa- 
tional fascism. » ."20 

The four-page Briefing Memo issued by NEA on Accountability and 
the Teacher is more moderate in language. It calls for "complete, 
not partial, accountability" and for an action program to "develop 
accountability measures to assess performance at every level of 
decision-making within school programs." 
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It decries the concept of accountability which seeks to assess a 
teacher's professional skill by testing students. An Information 
package developed to guide teachers makes several references to 
legislation. It emphasizes six elements (goal settings students^ 
programi staff, resources, governance) In assessing learning out- 
comes and calls for state legislation to sanction negotiations for 
policies and agreements at local level on these matters. It asks: 
What can teachers do "to prevent the enactment of unfair accounta- 
bility laws and ensure establishment of more equitable measures?" 
Actions advocated include: 

- Ensuring that responsibility of various groups 
(boards, administrators, teachers, parents, students) 
gets identified and assessed* 

- Working to have accountability measures take the 
form of state department regulations, which are more 
adaptable than legislation and easier to change and 
modify as new circumstances arise. 

- Seeking to prevent polarization of public thinking 
on education because of pushes towards systemization 
and standardization* The other side is humanization 
and greater openness. "Legislation should never be 
used to establish a single persuasion of educational 
thought." 

The NEA Briefing Memo urges teachers to engage in "appropriate 
collective action" in response to the implementation of "ill-advised 
accountability measures." "ill-advised" in this context applies 
to accountability plans which (a) use limited measures to assess 
effectiveness in education and/or (b) fail to apportion responsi- 
bility for outcomes and effectiveness of schooling among all the 
parties involved. 

New York State offers examples of the strong stance taken by 
teacher unions against accountability legislation. Resolutions sub- 
mitted by the United Federation of Teachers at the first annual con- 
vention of New York State United Teachers in 1973 included the fol- 
22 

lowing: 

- To support efforts to block the voucher plan at the 
federal level by the NEA, the AFT, the AFL-CIO, and 
other groups. 
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- To resist any effort to authorize school districts 
In New York State to sign performarr;e contracts. 



- To urge the Board of Regents to reject certain re- 
commendations of The Flelschmann Commission^ and to 
urge the State Legislature to refuse to enact them 
Into law. The offending recommendations pertain to; 
voucher plans for vocational education^ total com- 
munity control of New York City public schools, per- 
formance contracting, differentiated staffing, and 
merit pay for teachers. 

The Flelschmann Commission recommendations condemned by the UFT were 
viewed as containing '*a preponderance of anti-teacher recommendations, 
which would be deleterious to the cause of quality public education.'^ 

Thus, both the proponents and the opponents of mandatory state- 
wide testing and other types of accountability legislation voice the 
same arguments In support of their position; both contend that they 
act In the best Interest of quality of public education. 

Ambivalent Incentive Structures; 
The Fears of School Administrators 

It Is not only teachers and their unions and students and their 
families who find cause to attack testing and Its uses. Local schools 
and school district administrators have also evidenced wariness about 
how Information assembled for accountability purposes will be used. 
This Is particularly the case where there Is some ambivalence about 
the operation of Incentive schemes in which funding is keyed to levels 
of performance on tests. Are funds to go to schools that rate high on 
tests, Increasing their advantage over those scoring low? Or are they 
to go to schojls showing low on tests, without reward for those scor- 
ing high? Or is there some incentive scheme which can be devised to 
motivate all districts and schools to steadily improve performance? 

The ETS cites two examples la its 1971 survey of assessment pro- 
grams, illustrating the double-guessing which can go on when the 
incentive is acquiring more funds in a school district. 

The Commission worked for two years on the cost, quality and 
financing of education in New York State. Its Report emphasizes 
establishment of a state accountability system including all 
educational programs. 
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- One state is using reading test scores In a formula 
for determining specific sums of money to be allocated 
to school districts to provide reading specialist 
teachers. A school may suddenly find itself without 
funds for specialized assistance because it was pre- 
viously successful in Improving reading levels. 

- Under similar legislation in another state» funds are 
awarded to schools ranking lowest on common measures. 
Some school principals were talking of deliberately 
over-speeding test administration so school performance 
would not be up to the mark on tests, "if failure is 
to be rewarded > it is folly to be successful." 

"Beating the system" is an art practiced by many. Some consider 
it a survival skill. It is one of the learning outcomes of educa- 
tion which tests do not seek to measure but which can invalidate 
tests and the Judgments based upon them* Anomalous incentive struc- 
tures encourage such invalidation and are an obvious pitfall to be 
avoided in designing policies intended to increase the quality of 
education. 

Michigan has provided leadership in developing incentive struc- 
tures for allocating compensatory aid using Improvement in student 
performance as a prime criterion for the award of special funds. 

- MICHIGAN. Under Ch.3 of SB 1269 state funds were 
allocated to LEA*s on the basis of the number of 
pupils in K-6 found in need of substantial Improve- 
ment in basic skills. Statewide norm-referenced 
tests of reading and arithmetic were used to identify 
the educationally disadvantaged (defined as those 
falling below the 15th percentile). Schools with the 
highest concentration of such students received allot- 
ments. Ip effects there is a three-year performance 
contract between school districts and the state. 
Local schools worked with parents to translate general 
goals into performance objectives; local choices were 
made about the nature of instructional programs needed 
to achieve these; and local tests were devised to check 
attainment of objectives. Funding was based solely 

on pre- and post-tests. Schools received $200 for ench 
student achieving 75% of the performance objectives set^^ 
for him and were pro-rated for lower-level achievement. 

Funds were cut off or cut back where schools failed to Improve per- 
formance of students according to standards set by parents and 
teachers. While it is too soon to judge the longer-term impact of 
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this incentive scheme, initial results have been heralded as highly 
successful. 

Some features of the Michigan incentive structure are p'^ralleled 
in the March 1973 proposal by Rep, Albert Quie (R.-Minn.) to change 
the criteria for distributing federal funds under Title I> ESEA. 
This proposal would make educational rather than economic depriva- 
tion the basis for remedial funding and would use criterion-referenced 
tests as a measurement instrument. 

Basing incentive structures on labels or location of services 
has its pitfalls and anomalies. The Federal and Michigan examples 
above illustrate how a shift in label from "economically disadvantaged'' 
to "educationally disadvantaged" may change the flow of remedial fund- 
ing. Another example is provided by funding for special education. 
We noted earlier a trend towards moving chilri-jn from special to 
regular education classes. This shift in locus and labeling creates 
anomalies in states that base funding of the handicapped on the label 
or location of children rather than on their instructional needs: 

- State funds in many cases are distributed on the basis 
of the number of special education classes or the number 
of children in different categories. If handicapped 
children are in regular classes, they are not counted 
for extra aid in some states. However, other states, 
such as Tennessee and South Carolina, have new laws 
providing aid based on service rendered, not on labels 
or location of services. 

In addition to wariness arising from anomalous incentive struc- 
tures geared to test scores, school administrators have general mis- 
givings about how tests will be used in decision-making. Delegates 
at the 1972 meetings of the National Association of Elementary School 
Principals (NAESP) passed a resolution calling for principals to 
"resist individually" and to encourage their local and state groups 
"to resist any use of standardized testing for purposes which cannot 
benefit the child and may be harmful to his welfare." Some principals 
considered that testing puts a greater strain on principals and 
teachers than on children because educators fear that tests will be 
used as personal evaluation instruments. Few supported the use of 
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test data for comparing performance of school systems^ schools, 

classrooms, principals, or teachers, but a number of principals 

expected pressure from state legislatures and education departments 

to test for purposes of making such comparisons. '^Legislatures can 
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recognize only one thing-- statistics/^ said one principal. 



C. Evaluation and Decisiori?; ; 
The Adequacy of the Information about the "Adequacy of Education" 

Usln^ Test-related Data in Decision-making 

We have been discussing evaluation and decisions from the per- 
spective of multiple publics affected by decisions. We have seen 
that the search for social justice, efficiency and adequacy in edu- 
cation affects many people: students, parents, teachers, admlnls- 
trf.tors, among others. And we have discussed some of the fears and 
pitfalls to be anticipated in designing and implementing accounta- 
bility programs in which testing of student performance Is the major 
evaluation Instrument . 

Attention now shifts from fears about the use of information 
to the information itself. How adequate is the information generated 
under legislated statewide testing as a basis for judgments about 
quality and effectiveness and for decisions about the future develop- 
ment of education? Evaluation by testing, among other things^ is 
Intended to provide a basis for decisions "to correct deficiencies 
in the state education system," to "help districts (or schools) make 
Improvements in their weak areas," to guide educational planning, to 
"enhance the quality" of programs and schooling, and to "expand life 
opportunities and options of students." All of these imply what is 
stated most explicitly in the California legislation: the evalua- 
tion (based upon test scores and "related" data) is to be provided 
"so that the Legislature and individual school districts may allo- 
cate resources in a manner to assure the maximum educational oppor- 
tunity for all pupils." 
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Is the Information assembled and analyzed under statewide test- 
ing adequate In Itself to guide aiJocatlve decisions to achieve 
greater quality, equity and efficiency In schooling? On the basis 
of available evidence, It can be argued that It Is not. There Is a 
large gap between the ends espoused and the particular means utilized 
to assess their attainment. Even were we to overcome problems of 
testing and Interpretation of test scores, there remain difficulties 
In utilizing Identified evaluation Information In decision-making. 

Tests, Testers and Testing; Attack and Response 

There are problems with tests: what they seek to measure^ what 
In fact they measure, their reliability, their biases, their scoring 
reference systems. Tests, test makers and test users are under 
attack at the present time, even while standardized statewide testing 
is being mandated in many states* 

Major testing agencies are under fire. The prestigious Educa- 
tional Testing Service of Princeton, New Jersey was the subject of 
a Ralph Nader-style investigation in 1972, The expressed intent was 
to evaluate the evaluators, to look into "the functions of ETS, Its 
responsibilities to its consumers (students), its impact on the edu- 
cational system and charges of racial, financial, cultural and sexual 
discrimination,** Prior to investigation the Service was dubbed "an 

unregulated monopoly wielding a great deal of power but responsible 
26 

to no one." The director of this Nader-affiliated investigation is 

also directing a project for the National Student Association, The 

NSA project involves nine college-based national brainstorming sessions 

In which tests are held up to the light of "public accountability" and 

in which ETS, the American College Testing Program, and Psychological 

Corporation (which handles Medical College Aptitude Tests) are exposed 

27 

to public scrutiny, ^ 



For a discussion of testing problems, see pages 165-176, below, in 
the paper by Naomi White, 
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Some professional evaluators are to be found In the avant-garde 

of those urging caution In Interpreting test results. An ex-dlrector 

of ETS recently made a speech in which major emphasis was placed 

upon human aspects affecting test reliability; the transactions 
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among test makers, test givers, and test takers. He asks! 

How does one get the users of tests — espec ially those 
who use tests as Instruments for determining educational 
poli cy — to know enough about the innards of the tests 
they are using to have some clear idea of what the test 
scores are saying about what children are learning and 
schools are teaching? (Emphasis added] 

How far can problems associated with testing be mltlgnted by 

advances in evaluation technology? Some would argue that there are 

grounds for optimism^ Wayne Holtzman, for example, in an address 

entitled "The Changing World of Mental Measurement and its Social 
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Significance," identifies several pressures in American society 
consistent with and supportive of new developments in measurement 
and instruction. He notes among other things: the emergence of 
modular Curricula, using criterion-referenced testing for standard- 
ized mastery rather than normative testing for measuring individual 
differences; the contributions of NAEP (National Assessment of Edu- 
cational Progress) in test design and administration; and the develop- 
ment of high-speed electronic computers wh^ch can serve new patterns 
of Instruction and process large amounts of complex data. Even some 
of the problems of "individual privacy vs. the needs of social re- 
search data banks," he suggests, are technically capable of resolu- 
tion. As to cultural bias in tests, there is increasing recognition 
of cultural variability and some bare beginnings in collaborative 
translation, calibration and administration of psychological measures 
acrosis cultures and sub-cultures. 

Advances in techniques ^ in testing as in other* domains^ ave 
proceeding apace. They sharpen the need to confront the questions: 
What do we want to evaluate? What goals for education shall the 
masurement techniques serve? And who shall decide^ by what means, 
the goals and the uses of evaluation? 
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"Detertninaats*^ of Scholastic Achievement: 
The Need to Examine Assumptions 

Let us suppose that advances In the field of measurement can 
considerably reduce problems of tests and testing. Let us suppose 
that we can sensibly lessen dangers of misinterpretation of pub- 
lished test scores by furnishing clear caveats and associated In- 
formation. Let us suppose that we can allay fears of Invasion of 
privacy, reduce the temptation to falsify responses, and mitigate 
problems of ambivalent Incentives structures. EVen If we posit 
progress on these dimensions, there remain some questions about 
utilizing the Information generated under testing programs as the 
primary basis for policy decisions. 

The Infoirmatlon assembled under statewide testing programs, as 
we noted earlier, consists of basic test scores and "related data." 
The latter are considered to have some explanatory power; they In- 
clude variables referred to In the literature as "determinants of 
achievement." When policy decisions are contemplated that rest 
heavily upon test-related Information, It Is Important to examine 
possible latent assumptions about "determinants of achievement." 
Examining these surfaces further problems to be addressed In design- 
ing and Implementing policies to serve quality, equity, and effi- 
ciency In education. The assumptions, stated broadly for discussion 
purposes, are of the following order: 

a) That the variables examined In relation to test 
scores (primarily selected Input measures) determine 
or explain achievement more significantly than vari- 
ables not examined. 

b) That educational policy can significantly increase 
achievement test scores by influencing these identi- 
fied input variables. 

c) That performance on mandated tests is the primary 
indicator of quality of education. 

d) That educational policy towards K-12 schooling sig- 
nificantly affects employment and income and thus 
post-school quality of life. 
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e) That analysis of the part (performance In basic 
skills) Is appropriate for decision-making affect- 
ing the whole (statewide quality of education). 



As they stand » unqualified, these assumptions suggest a degree 
of assurance for educational decision-making which contrasts sharply 
with the ambiguity of the research evidence. 

Ambivalence and Gaps in Research Findings 

There is a considerable literature on determinants of scho- 
lastic achievement which may be summarized thus: Socio-cultural 
Influences, inside and outside a formal educational system, affect 
aspirations, performance and scholastic achievement of students. 
However, there is some disagreement and a great deal of ambivalence 
in discriminating which factors are most important, the degree of 
their significance, and hence the Implications for educational 

policy. A brief review of research will illustrate this generali- 
30 

zation. 

There are recurring controversies about the relative influence 
of genetic and environmental factors upon ability. Even the stron- 
gest supporters of the genetic determinant tradition admit to some 
environmental influence on ability. There is a substantial body 
of international research that rejects the idea of some fixed 
"pool of ability,'* suggesting that ability is not a fixed quantity 

but is strongly Influenced by environmental factors inside and out- 
31 

side schools. Thus, the major research alliance called The 
International Project for the Evaluation of Educational Achievement 
(TEA), after some six years of work involving twelve countries, 
emphasized the relationship of scholastic achievement in mathematics 
to organizational factors (school organization, curriculum, instruc- 
tional methods), and ''sociological, technological, and economic 

32 

characteristics of families, schools and societies." This general 
finding is confirmed in a host of studies. So is the interdependence 
of the variables — and therein lies a problem for deriving policy 
Implications . 
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Thus far most attention has been given in research to the Influ- 
ence of environmental factors outside the school: socio-economic 
composition of the population} family background factors, such as 
parental education, paternal occupation and Income, family size, 
ethnicity; community and neighborhood factors. In additioni large- 
scale studies associated with major federal policy interventions in 
the sixties have sought to assess the influence of resource inputs 
(funds, teachers, facilities, etc.) upon test scores. The intent 
was to furnish an information base for policies to equalize educa- 
tional opportunities — a policy goal originally interpreted as equal- 
izing access to education but increasingly conceived as equalizing 
opportunities for scholastic achievement. 

As for variables within the school learning environment, there 
has been a lesser but growing effort by social scientists to identify 
those that are significant for measured achievement. An examination 
of the United States literature suggests that, of a range of variables 
connected with school organization, teacher characteristics and stu- 
dent characteristics, three are of particular significance: the 
attitudes of teachers, the composition of student peer j^^roups, and 
the self-concept of students. 

The closer we move to the locus of teaching and learning, the 
greater is our ignorance of how being in school affects what and how 
students learn and what they do with it. It is easy to find agree- 
ment that the relationships of students, teachers and teaching/learn- 
ing styles ("what happens in classrooms'^ are of fundamental impor- 
tance to education and whether schools are more or less educationally 
effective. Yet there is a dearth of policy-relevant research examin- 
ing "process" variables and their relationship to learning. We have 
large banks of data which Inform us of the relationship of selected 
Inputs and test-defined outputs at school, district or state level; 
but we lack discriminating information about the relationship of 
particular resources, environments, and human interactions to the 
learning of particular students. 
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Research evidence gives little basis for assurance that we know 
which variables most affect achievement of what types of students, 
In what dimensions of learning, under what complementary circumstances • 
At the present time there is no generally accepted social-psychological 
theory of learning which would provide a basis for analyses of the 
efficiency of schools in tenns of student achievement and its deter- 
minants. The messages relayed by contemporary research to the domin 
of deoision^making are ambivalent and cautionary^ rather than assured. 
Without better knowledge of the system inputs and process variables 
which most affect learning outcomes y we are ill-equipped to influence 
these outcomes by policy and planning. 

But what about resources? Do not more resources raise performance 
levels? Again the evidence is ambivalent • The Equal Educational 
Opportunity Survey (the Coleman Report), the many reworklngs of its 
data, Project Talent, the Plowden Report (U.K.) and the recent and 
controversial study by Jencks and others, all suggest that though some 
resources and policies do sometimes bear relationships to cognitive 
achievement scores, the general effect of variations in resource Inputs 
(funds, facilities, teachers, etc.) is either slight or unpredictable.^^ 
This is not to say that resources make no difference; it Is to say that 
we do not know under what conditions particular combinations of re-- 
sources^ used with what particular target groups^ along with what 
complementary conditions^ may influence an individual's eventual 
level of competence in reading ^ verbal and calculating skills. 

This conclusion does not offer much guidance for policy decisions. 
There are features of research-generated inforraatlon that mitigate 
against policy relevance. The broad evaluations of major Interven- 
tions in education, deliberately Intended to guide public policy, fall 
short of serving that purpose well. This is partly because of the great 
complexity of interdependencles among variables examined; but Is 
also occasioned in part by dealing largely in variables reriote from 
the locus of learning, and by utilizing macro-data that average out 
significant variation at the teacher/learner level. On the other 
hand, less publicized and more fragmentary micro-level studies of 
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schooling do more often deal with variables proximal to the locus of 
learning; but they generally fall to take account of costs and they, 
too, fall short of policy relevance. 

Problems for Policy Inference 

Given the ambivalence of research findings and the constraints of 
the data, any policy Inferences dravm must be examined with utmost care. 
An example will Illustrate the problems. A major research report, 
having reviewed Input-output studies of school effectiveness and Iden- 
tified deficiencies In design and data, affirms that the evidence Is 
Inconclusive: The research falls to show that school resources do 
affect student outcomes; but It does not show that they do not; It Is 
unable to Identify what -particular resources should be provided to 
students. Despite recognition of the limitations of the research 
reviewed, the report authors state: 

(Input-output research] has yielded one Important policy 
Implication . The resources for which school systems hav 3 
t raditionally been willing to pay a premium — teachers' 
experience, reduced class size, and teachers' advanced 
degrees — do not appear to be of great value . Inexperienced 
teachers do not appear to produce students whose outcomes 
are significantly worse than the outcomes of students whose 
teachers are experienced, other things being equal. Simi- 
larly, students whose teachers have advanced degrees or who 
are in small classes do not do better, other things being 
equal, than students of teachers lacking advanced training 
or attending large classes. [Emphasis added] 

The policy Inference is that efficiency in schooling can be raised 

by increasing the student-teacher ratio and by using teachers with 

lass formal education and less teaching experience (same output, 
smaller Inputs of salaries). 

The shaky basis for this policy inference will be more apparent 
if we sharpen the nature of the finding. More accurately stated 
it is of this order: Student/teacher ratios on the average within 
X range show little significant Impact upon scores of students on 
standara. /^d achievement tests of lower cognitive abilities. Three 
considerations must then be noted which challenge the generalized 
policy Inference that reducing teacher-related Inputs will Increase 
the' efficiency of schools. They concern the nature of the output 
indicator, the aggregation of data, and the nature of the input 
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indicators used in input-output studies. 



(1) The outcome indicator utilized often reflects only lower 
cognitive skills» using tests which have imperfections even for that 
purpose. The measures emphasize, for example, whether children know 
how to read, rather than whether they do read, the motivation to read, 
what they read, and how they integrate what they read into thinking 
and doing. They disregard higher cognitive skills and affective 
development. Does the student/ teacher ratio influence learning in 
any of these broad areas? We do not know; we think it might; we 
cannot conclude from input-output research that it does not. 

(2) The aggregation problem. Input-output studies typically 

use input data aggregated to school or district level and thereby 

average out significant variations. Several studies confirm that 

about 65-70% of the variation in student achievement scores occurs 
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between pupils in the same school. Thus, when resource or other 
"determinants" data are aggregated to school level they cannot 
account for more than one-third of the variation in scores; at 
district level the variation that can be accounted for is even less. 
Does the student/ teacher ratio affect achievement scores? We do 
not know; we think it might; we cannot , conclude from input-output 
research that it does not because we have no information about the 
relationship between particular ratios (or the experience and training 
of particular teachers) and the performance of specific groups of 
children at the intra-school level where most of the score variation 
is found, 

(3) The remoteness of the input indicators utilized from the 
locus of learning activity is a prime constraint. Me need to know 
how resources are used as well as their quantity. In the case of 
teachers, years of training and experience may be less significant 
than the nature of the training and the experience, and the way 
these resources are utilized within a school and a classroom. We 
need to know the relationship between different patterns of resource 
utilization at class level (including use of teacher and student 
time) — the Interaction of students, teachers, other resources, and 

^ teaching/learning styles. Does the student /teacher ratio, teacher 
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experience, and training affect student learning? We do not know; 
we think they do; we cannot conclude from input-output research 
that they do not, because the research does not utilize indicators 
that are sensitive and proximal enough to the activity of learning. 
A finding of the Pennsylvania Quality Assessment Program emphasizes 
the point: 

- PENNSYLVANIA Educational Quality Assessment Program 
collected data for 47 condition variables and found 
that they did not explain 51% of the variation in 
standard performance on a range of diverse goal 
measures. ''The most likely explanation appears to be 
that process variables, the things students and teach- 
ers do in the classroom and other school settings, are 
important contributors to the variability of students 
in goal i^arformance measures* These variables, being 
modifiable by the school provide hope for the pos- 
sibility of improving the quality of education." 
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Countering Some Assumptions 

From this discouraging sortie into the research findings, we now 
return to countering the assumptions apparently underlying the use 
of information assembled under statewide testing for purposes of 
guiding decision-making in education. These points can now be made: 

- Variables selected for examination in statewide test- 
ing programs do not explain achievement more signif- 
icantly than variables not examined. 

- Selected variables fall heavily in the domain of 
influences external to the school. These influences 
are not amenable to change by K-12 educational 
policy alone: they require coordinated social 
policies in such domains as income and employment — 
and no doubt a sophisticated approach to educating 
adults. 




- In-school variables are amenable to treatment by 
educational policy and they lie within the locus 
of control of the school. However, with rare and 
localized exceptions, they are given scant attention 
in evaluating the adequacy, effectiveness and effi- 
ciency of schools. 

- There are correlations between measures of cognitive 
scholastic achievement and employment and income 
after completion of schooling. However, current 
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research does not support the assumption that these 
elements of ''quality of life" are readily amenable 
to regulation by educational policy alone* 

- There Is considerable doubt to be cast on assuming 
that we can rely heavily upon test scores as the 
major measure of the outcomes of schooling and the 
quality of education. At best these data may provide 
a valuable Indicator of the status of achievement 
In limited dimensions of human development* But> 
without attention to other Important dimensions and 
to relative costs, there Is no basis for assessing 
the complementarities and trade-offs to be considered 
In planning the future of education. 

We have a vast amount of evidence concerning what schools can- 
not do. They cannot assure employment. They cannot assure Income. 
They cannot assure quality of life after completion of schooling. 
These matters are uot amenable to guarantee through educational 
policy directed at compulsory schooling. They depend heavily upon 
coordinated, complementary policies In several dimensions of national 
life. 

What can the schools accomplish? What are they good for? How 
do they affc^ct the lives and learning of those who spend Increasing 
years pf tholr lives there: students, teachers, administrators? 
These are questions that cry out for attention In the process of 
educational policy planning. Our quest to define the Information 
needs for decision-making is futile without examination of the edu- 
cating purposes of schools and unless we emphasize areas where edu- 
cational policy can make a difference to the achievement of those 
purposes. The Issues associated with the purposes of schooling are 
critical and they deserve the special discussion accorded them in 
the ensuing paper. For the present, let us re-emphasize areas in 
which educational policy can make a difference — and, therefore, for 
which accountability may be legitimately required. 

Educational policy can make a difference to life Inside schools. 
It can affect the variables at work inside the classroom* We could, 
if we choose, seek to improve the quality of life and learning of 
those who spend large proportions of their lives in schools. If 
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we were to seek indicators of "quality of education" ranging beyond 
standardized test scores, we might begin to identify variables sub- 
ject to educational policy which would indicate quality of life in 
schools . This would parallel in some ways the shift in attention at 
societal level from total preoccupation with economic indicators of 
national "progress" to broader social indicators of the state of the 
nation — from measuring economic growth to seeking to assess quality 
of life. Such a shift in emphasis might blur some of the false 
distinctions which make formal education (all 10-20 years of It) a 
prelude or a means to life and living, rather than an integral part 
of life and living. It might enable us to move more easily in and 
out of those two parts of our world which have been labelled "work" 
and "education/' 

The broadening of perspective implied in seeking a range of 
indicators of "quality of life in schools" Is not a retreat into 
the pleasure principle: happiness for its own sake whether there 
are learning outcomes or not. On the contrary: the quality of life 
and the quality of learning in schools are intimately related* The 
quality of life perspective would require greater attention to the 
locus of teaching/learning activities in schools and to the variety 
of ways in which different human beings interact with each other 
and with their environment (ind resources in the process of learning. 
Thus we would come closer than we are now to understanding how the 
educating purposes of schools might be achieved — more equitably, 
more effectively, more efficiently* And this is the expressed 
Intent of the legislation examined, 

Information for Decisions: 
What Kind of More is Better?. 

Data for decision-making, it can be argued, are never adequate, 
never sufficient, only more or less so. It is tempting to conclude 
that what we need of any kind of information is "more,'' that when 
we have more information we will be better off, will make sounder- 
based judgments than when we have lesi . But, in data collection 
as in education, more is not necessarily better. In information 
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collection as in education, the scarcer resources become the more 
important are the trade-offs in allocation. Expenditures on massive 
data collection in one area may preclude generation of data in 
others or the quality of analysis that can aid policy-makers • Major 
new banks of data on limited aspects of measured learning may be 
bought at the price of information on a wider range of indicators 
of school performance and the range of information which can make 
analysis policy-relevant. There is a real danger that decisions 
may then be made which distort learning opportunities and run counter 
to the intention of increasing the overall quality of education. 

To be useful for decision-making information has to be policy- 
relevant. More information will not provide a better basis for 
policy Judgments if it serves only to confirm what we know: that 
some educational problems are not school problems but societal prob- 
lems whose melioration requires concerted social policies. More 
information will not be policy-relevant if the data are so aggre- 
gated that they average out significant variation in achievement 
and fail to discriminate how in-school. In-class environment and 
interactions affect student learning, behaviors, and the quality 
of living in schools. More information will not be policy-relevant 
if outcomes and effectiveness are not related to costs, and if costs 
as well as outcomes are narrowly construed. 

In data collection as in education we have to ask: What kind 
of more is better, for what purposes, for whom? There are no one- 
shot answers to those questions any more than there is a single 
legitimate measure of the quality of schools. These questions have 
to be confronted in a continuing process of policy planning which 
engages relevant actors in addressing issues of alternative goals, 
evaluative criteria, strategies, and actions for the development 
of "quality education. 
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PART FOUR 

BEYOND THE LEGISLATION; THE POLICY-PLANNING PROCESS 



An Exploration 



Contemporary statewide testing/assessment legislation functions 
as part of the traditional political process which formalizes public 
decisions about education. At the same time It has the potential to 
change that process. In several states the legislation requires the 
development of new mechanisms and procedures for decision and action. 
These deserve close examination both for their Impact upon traditional 
modes of administration and for their success In dealing with problems 
and Issues confronted In the decision-making process. 

As we saw earlier, a major difficulty arises from the Inadequacy 
of contemporary research and Its findings to guide allocatlve deci- 
sions that will Increase efficiency > effectiveness, quality, and 
equity In education. If we were to wait, however, for definitive 
research findings before acting, probably little action would be 
taken. The absence of sure-fire strategies and guarantees of suc- 
cessful outcomes complicates the process of decision-making. This 
complication, In some measure, will probably always be present. It 
emphasizes the precarlousness of relying unduly upon any single Indi- 
cator of problems or any single measure or criterion of success In 
resolving them. It underscores the need to design and Implement 
policies on the basis of continuing, critical appraisal of alternative 
goals, strategies, tactics, and their potential consequences. It 
also suggests the importance of devising policy instruments that can 
be flexibly applied to varying circumstances and adjusted as evidence 
emerges from evaluation of policy implementation. The process by 
which these various activities are facilitated is what we call the 




policy-planning process. It is the focus of the ensuing discussion. 



Analysis to this point has been based upon study of legislation, 
litigation, evaluative research and associated documentation* To 
analyze processes In the states in which alternative goals, policies 
and policy instruments are evaluated, decisions made, and action 
initiated and monitored would require substantially different docu- 
mentation supported by field study. That being so, this discussion 
of aspects of the policy-planning process is exploratory— a prelude 
to investigation, not a report upon it. We will consider first the 
relationship of goals, evaluation and resource allocation in the 
overall decision-making process; then the scope of action and the 
range of participants involved in planning within that larger pro- 
cess; and finally, the nature of changing conceptions of planning 
functions, roles and styles. 



A. Evaluation, Resource Allocation, and Goals 

We are concerned in this exploratory discussion with a con- 
tinuing process wherein goals are generated and examined, objectives 
specified, evaluations undertaken and interpreted, and policy alter- 
natives analyzed to guide decisions affecting the future of education 
and its publics. 

Evaluation and Resource Allocation 

Thus far discussion hafi emphasized evaluation aspects of that 
process — a major feature of the statewide testing legislation. Some 
related problems and questions have already been identified. Some 
of them can be anticipated and dealt with more readily than others 
during the planning process. We noted, for example, the need to 
examine problems of tests, testers and testing; alternative Incentive 
structures and their impact; ways of minimizing misinterpretation of 
evaluation information. We noted also the stake of multiple interest 
groups in addressing these and other less tractible issues; for 
example, the sensitivity of evaluation data and the distribution of 
responsibility for outcomes. In every case where evaluation was 
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under attack the primary concern was with the ways In which evalua- 
tlon-by-testlng was being or might be used In decision-making. 

In the last analysis, Information generated under legislated 
programs is Intended to guide decisions about the allocation of 
resources to promote Increased quality, equity > and efficiency in 
schooling* There are perhaps more analytic tools available to 
examine efficiency than quality and equity, but even these have not 
been skillfully used in the evaluation of schools* 

How shall we allocate among competing ends scarce means which 
have alternative uses? This is the classic problem addressed in 
economics. Addressing it in the domain of education requires an 
understanding of learning processes and purposes as well as economic 
analysis; understanding its resolution in practice requires atten- 
tion to political as well as economic rationality. Nevertheless, if 
we do not address that question, then we belie the claim to under- 
take what the legislation terms "objective evaluation" of the "ade- 
quacy" and "efficiency" of public schooling. At a minimum, address- 
ing the question requires some clarity about "competing ends," about 
"scarce means," and about "alternative uses" of resources in schools, 

We have seen that evaluations of school effectiveness often 
take "ends" as given and pay little attention to alternative utiliza- 
tion patterns of "scarce means" in the teaching/learning process ♦ 
Assessment of the relationship between ends and scarce means then 
becomes a travesty of the rational analysis of alternatives which 
could make a major contribution to the basis for policy judgments, 
The infonuation assembled under statewide testing programs, legis- 
lated or not, is subject to the same range of criticism. Heavy 
reliance upon Information generated in evaluatlon-by-testlng can skew 
attention, allocations, and activities in schools towards what is 
measured and rewarded to the detriment of the range of complementary 
and competing educational purposes of schooling. 

What are the penalties of emphasizing analysis of the part in 
making allocatlve decisions affecting the whole? What are the 
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opportunity costs, what is the learning foregone if we place the 
weight of resources on some dimensions of human development rather 
than on others? We are here entering the rough terrain of goal and 
value analysis and the problems of assessing the unintended conse- 
quences of achievement of some targets for the achievement or non- 
achievement of others. 

Questions of Goals, Priorities, and Objectives 

Discussion of the ^problems associated with evaluation largely 
bypassed questions of goals, priorities and objectives, save where 
examination of assumptions brought us to the thorny questions: What 
are the schools good for? What do we mean by quality of education? 
Yet goal-related questions are basic to the policy -planning agenda. 
What goals, whose goals for education? What kind of education? 
for what purposes? for whom? where? when? how? And who shall 
engage in addressing and answering these questions? on what grounds? 
in what process? requiring what information? and what competencies? 
These are fundamental, universal, continuing questions that are 
answered explicitly or by default in all countries and In all time 
periods. They are politically salient in the United States at the 
present time, but they are rarely confronted and carefully analyzed 
in the traditional arenas of- public decision-making. 

The broad issues which impinge upon the choice of goals and 
priorities in education are also fundamental and enduring. They 
involve, among other things, the balance to be sought in education 
between uniformity and diversity; between homogenization and indi- 
vidualization; between individual development and socialization; 
between protecting minimal rights and protecting and facilitating 
freedom of choice and action; between what various groups desire 
and what society will tolerate in the way of alternatives within 
schools, alternative schools, and alternatives to schools. The 
balance between these elements cannot be resolved once for all. 
Rather, we have to seek a continuing awareness of the state of 
balance and tradeoffs, lest we swing to extremes of emphasis which 
distort the equity, efficiency and quality we seek in education. 
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These goal-related questions and issues can and must be addressed 
in relation to the current wave of statewide testing/assessment legis- 
lation and practices. Hence the critical importance of the companion 
paper which follows. It considers the implications of the emphasis 
on minimal skills (translated as reading, writing, and computation) 
in the statutes of several states. There are several possible con- 
ceptualizations of "basic skills'* — alternative and complementary to 
the 3R's interpretation — to be derived from contemporary learning a.'^d 
social theory • Each incorporates an understanding of the nature of 
human learning and development. Each involve;-; beliefs about the 
social role of education. The various ways of interpreting "basic 
skills,'' if explored and implemented, could have profound consequences 
for the viability of local versus state determination of educational 
goals, priorities, and objectives; for what these might be in dif- 
ferent communities; for the form that evaluation of their attainment 
might take; for which agencies might legitimately be held responsible 
for ensuring the acquisition of particular skills; and for informing 
discussion in public and political forums where decisions about re- 
source allocation are made. 

It is apparent that questions of "what education — for whom — 
where — when — how" have complex ramifications. The notion of basic 
or minimal skills alone opens a Pandora's box of issues — including 
those of balance between guaranteeing minimal rights and protecting 
freedom of choice and action. 

The primary concern here is not^with the goals and issues them- 
selves, but with the nature of the continuing process in which ques- 
tions and issues of goals, priorities, objectives, evaluation, im- 
plementation are addressed systematically as a basis for policy 
decisions. That process may be conceptualized in terms of the span 
of inquiry and action that it encompasses and the range of actors 
that it engages. 



ErJc 136 



B. Dimensions of the Planning Process; 
The Span of Act ion and the Range of Actors Involved 



The programmatic approach to educational planning observed In 
many states and in the operations of federal agencies tends to encourage 
partial planning and analysis. It Incurs concomitant disadvantages 
for integrated decision-making. Thus, in the domain of assessment 
programs, there is evidence of considerable overlap and lack of com- 
munication in some states. There is duplication of effort and there 
are gaps in effort. There is little evidence that cross-impacts, cross- 
purposes, and complementarities of piecemeal programs Intended to in- 
crease the ^'quality of education** are examined comprehensively in a 
continuing policy-planning process. 

For discussion purposes we might posit a span of inquiry and 
action in which the scope of Interest and activities broadens thus! 



Testing — Evaluation — Assessment — Program - Comprehensive — Policy 

Planning Planning Planning 

SPAN OF INQUIRY AND ACTION 



Technical technicalH- 
apolitical political 



At one end of the stylized continuum we have a testing activity, 
apparently technical and apolitical (but not regarded so by all publics 
of education) ; at the other end of the continuum we posit a policy- 
planning process which encompasses testing and other activities, and 
which deliberately seeks to marry technical analysis with policy deci- 
sions. The testing end of the continuum takes goals as given, and is 
likely to have operational objectives relating to narrowly specified 
dimensions of human development. The policy-planning process seeks to 
deal with analysis, including goal analysis, affecting the whole. It 
includes consideration of alternative policy decisions affecting the 
future of education (as contrasted with, say, programmatic decisions 
about the future of reading skills) • 
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Between the testing and the policy-planning activities, we posit 
several others. Evaluation is a broader category than testing, It 
encompasses a wider range of instruments for appraising a wider range 
of performance. And it includes attention to costs and to process as 
well as to outcomes. Assessment , as used in "needs assessment,^' in- 
cludes the elaboration of goals, their translation into performance 
objectives, and the evaluation of the gap between what is occurring 
(as evaluated) and particular objectives. Assessment may be general 
(pertaining to all aspects of schooling and learning) or particular 
(relating only to student outcomes and/or particular learning areas 
ouch as 3R's). It may or may not be part of a continuing policy-planning 
process. Of itself, it does not necessarily imply a continuing process; 
nor does it incorporate the generation of designs for remedying defi- 
ciencies, improving programs, and presenting costed policy alternatives 
for decision-making* 

Both program planning and comprehensive planning Involve a con- 
tinuing process which includes some form of assessment, plus the pre- 
paration of costed alternatives for future development of education 
and monitoring the implementation of alternatives chosen. In program 
planning the analysis emphasizes specific aspects of the development 
of education and there is no necessary implication that the activity 
takes into account interactions with other programs. Comprehensive 
planning , on the other hand, requires integrated analysis of various 
programs with attention to tradeoffs and complementarities among pro- 
grams and among strategies of action. By giving explicit attention to 
coordination of activities, comprehensive planning seeks to avoid the 
overlaps and gaps that characterize uncoordinated program planning. 

Both "program planning" and "comprehensive planning" refer to 
systematic rational analysis; neither deliberately attends to the 
politics of planning. Policy planning , as the concept has been developed 
at EPRC/Syracuse, deliberately seeks to marry systematic technical anal- 
ysis with concern for the dynamics of decision-making, notably by giving 
explicit attention to the nature of participation in the process and 
to the ways in which information generated In the process can be used 
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by policy-makers and the multiple publics of education affected by 
their decisions. 

In order to achieve some conceptual clarity, we have distinguished 
various terms according to the span of inquiry and action they involve. 
The ascribed definitions, however, are not neatly maintained in 
practice. People may speak of "assessment" and "evaluation" when 
in reality all they are doing is "testing" student achievement. 
"Assessment" is sometimes used to describe programs which, upon 
examination, involve only impoverished versions of goal analysis. 
The term "comprehensive planning" is sometimes applied to activities 
which at best coordinate only a few programs (for example, those 
generated by federal monies and reported according to federal guide- 
lines) . In any given case, therefore, it is important to examine 
the actual range of questions addressed and the actual nature of 
activities undertaken rather than to assume the span of inquiry and 
action from the labels ascribed to them. 

If testing is not part of an ongoing policy-planning process, 
then we are likely to find partial analysis substituting for overall 
analysis, with testing targets taken as surrogates for the goals of 
education and test scores taken as the valid measures of quality In 
education. If, on the other hand, 'testing is an integral part of a 
continuing policy-planning process, then we can at least posit the 
possibility that it will serve rather than determine goal and strategy 
priorities in educational policy. 

The Range of Actors 

Who participates in the policy-planning process? Here again 
there is a span of possibilities. The range of possible actors 
might be arrayed thus: 
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RANGE OF ACTORS 



Technical Multiple 
expertise publics 
emphasis emphasis 



The more testing and planning are viewed as "purely'' technical activ- 
ities » the more they are likely to be considered the preserve of 
"experts/' The more they are seen as not "purely" technical, not 
apolitical (that is, the more they are perceived as highly relevant 
to political decision-making), the greater are likely to be the pres- 
sures by multiple publics to participate in the policy-planning pro- 
cess » 

Similar pressures are encountered in the process of designing 
and seeking to enact legislation* A recurring complaint of teachers, 
for example, is that they are asked to participate, not in designing 
policy, but only at the level of implementing requirements which they 
perceive threaten both their interests and the "quality of education," 
This does not imply that there are no ways in which Interest groups 
can influence decisions • Lobbying, boycotting and other forms of 
action are traditional and effective forms of politicking in educa- 
tion as in other domains. When engagement takes this adversary form 
at a late stage in decision-making, however, it suggests poor development 
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of mechanisms for systematically involving various groups and taking 
account of their views within the continuing process of policy- 
planning « 

Action and Actors: A Matr ix 

We might pursue the development of an heuristic device. The 
matrix on the next page brings together conceptions of the span of 
action and the range of f^^^ors discussed above and suggests broad 
patterns yhich might be fourid in the states. 

Pattern A reflects a situation where activities related to 
testing and evaluation programs are seen as the domain of action 
of technical experts and do not encompass goal and strategy analysis 
or involve broad participation. The diametric opposite pattern, 
Pattern characterizes the case where there is a comprehensive 
policy-planning process and deliberate efforts to involve multiple 
publics in the process. Intermediate Pattern B suggests situations 
where various publics are involved but only in restricted dimensions 
of policy-planning (perhaps in examining alternative means, but not 
alternative goals; perhaps in generating various goals, but not in 
examining priorities, consequences, or cross-impacts). Pattern C 
suggests a comprehensive but technicist approach to planning where 
the process is largely dominated by technical experts and managerial 
efficiency concerns, with negligible attention to participatory modes 
and the development of administrative styles to accommodate them. 

It might be possible to roughly plot various points in the matrix 
that approximate the position of particular states at the present 
time — and perhaps over the span of the past five years. This would 
suggest, among other things: some trends to monitor; some lead states; 
states demonstrating extremes of the stylized patterns; dimensions of 
innovation in roles and operating styles in state education agencies; 
development of capabilities at state level which will be of crucial 
importance the more there is a move to revenue-sharing and a devolu- 
tion to sea's of that part of the heavy burden of planning, evalua- 
tion and administration thus far borne largely by the U.S. Office 
of Education. 
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TABLE 2.14 



f>ATTERNS OF EDUCATIONAL PLANNING AND EVALUATION 
WITH VARYING SPAN OF ACTION AND RANGE OF ACTORS 



Involvement of multiple 
publics, but restricted 
span of action 



Involveoient of multiple 
publics in comprehensive 
policy -planning process 



Restricted span of action 
and restricted range of 
ictors 



Broad policy -planning but 
restricted involvement of 
actors 



Testing — Evaluation Assessment — ^'"^^^"^ Comprehensive — Policy 

Planning Planning Planning 
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I I I -SPAN OF ACTION 1 



Webster /E PRC 
M»y 1973 



Action, Actors, and Planning Capabilities 



There are growing pressures upon planning units in state educa- 
tion agencies to expand their span of action and to increase the 
range of people Involved In the expanded activities. To what extent 
is a planning capability being developed to meet these challenges? 
As relatively young planning units seek to confront major new respon- 
sibilities, what kinds of problems do they encounter and to what 
extent are viable solutions being found to them? What kinds of plan- 
ning competencies are needed to meet new demands? To what extent 
are they being developed and by what means? These and related quesr 
tions cry out for attention at the present time. Without reliable 
answers to them it is not possible to answer another important ques- 
tion; What policies, at state and federal level, might strengthen 
the planning function in the states in the service of developing 
"better schools"? 

From observation and discussions in various states and from 

such documents as are available It is possible to sketch some ele- 
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ments of the situation of planning units. There are variations 
within and among states in dealing with the action/actors spectrum^ 
in planning and there are variations, in the functions, capabilities, 
styles of operation, and perceived status and legitimacy of educa- 
tional planning units, A few state agencies appear to have institu- 
tionalized planning departments, with a firm base of support, and 
have accumulated some experience in developing a planning process 
(as contrasted with "making plans"). Most planaing units, however, 
have been established since 1970 (when Title IV, Section 402 ESEA 
funds became available). Many are suffering the growing pains of 
establishing a planning capability, with precarious funding and 
uncertain status in the agency, and have barely begun to confront 
the complexities of designing and implementing a viable planning 
process • 

In a few states there has been experimentation in introducing 
participatory mechanisms at various levels of decision-making and 
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in a variety of planning activities. In many ocher states partici- 
patory experience is more restricted in type and in depth of involve- 
ment. Thus, many states have orchestrated elaborate exercises in 
setting state goals for education, involving large numbers of citi- 
zens; but it is not clear whether mechanisms are being developed 
to facilitate a continuing participatory process involving a broader 
range of activities. 

The political, economic, and general social climate varies 
among states, presenting \Ufferent conditions for educational policy- 
planning which must be taken into account. Nevertheless, many of 
the important policy issues and many of the problems encountered in 
planning are substantially similar across states. What is learned 
in some states about policy analysis, problem solutions, alternative 
development strategies, participatory mechanisms and their efficacy, 
and competency development is likely to be useful in others* 

This depiction of variations in experience and commonalities 
in problems is drawn with the broadest brush* There is little in 
the way of well-documented critical appraisal of planning activities 
and needs in the states; and mechanisms for generating and sharing 
policy analysis and planning experience are poorly developed* 

We know more at present about what the polity expects than 
about planning capabilities to meet the expectations. We know also 
that prospective federal policy changes are likely to increase pres- 
sures on the planning function in state agencies* Decentralization — 
of problems as well as of administration — will increase the already 
heavy demands on planning capabilities in the states. At the same 
time designs for revenue sharing point to the likely demise of 
specific federal support for the educational planning function at 
a time when strengthening it is crucial to the development of "better 
schools 

If policy judgments, both at federal and at state levels, affect- 
ing the strength of planning capabilities In the states are to be 
based upon sound evaluation, time is short for undertaking a needs 
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assessment In the area of state educational planning. The kind of 
appraisal needed should Include attention to these dimensions: 



- Span of Action ; Where do planning activities fit on 
the continuum from narrow technical functions to com- 
prehensive policy planning? What levels and kinds of 
education are Included (public, private, K-12, voca- 
tional, post-secondary) ? 

- Information : What kinds of information are most essen- 
tial for what kinds of decisions? How is available 
information used? Who has access to it? What informa- 
tion is needed but not available? How are overlaps of 
federal and local information needs handled? 

- Relationships with Other Group s; What are the relation- 
ships of the planning unit with relevant groups at state, 
federal and local levels in terms of (a) technical co- 
ordination, (b) competition for resources, (c) political 
Influence, (d) accountability? 

- Competencies : What kinds of planning capabilities and 
personnel competencies are needed for (a) technical- 
analytic work, (b) managing the participatory process, 
(c) integrating these two? 

- Participation : In states that claim to be developing 
participatory mechanisms, how are the roles, functions, 
and needs of participating publics handled, especially 
in terms of the above dimensions: span of action, 
information provided and generated, Inter-group rela- 
tionships, and competencies? 

- Legislation and Guidelines ; How is the functioning of 
the planning unit affected by the presence or absence 
of strict, prescriptive legislative or administrative 
mandates? 

An appraisal of this order could provide an overview of state planning 
capabilities and identify planning needs* We would then be in a 
position to make better judgments about policies to strengthen state 
education agency units upon which devolve heavy responsibilities for 
planning the effective development of education- 
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C» ChanK^lng^ Conceptions of Planning Functions and Roless 
An International Perspective 



It Is worth considering changing viewpoints In educational plan- 
ning In countries having substantially more experience In this domain 
than the United States. Their experience can bring a fresh perspec- 
tive to the questions now being raised In the states of this country 
where planning Is In embryonic stages of development. Moreover, it 
may help us to distinguish the temporal from the continuing Issues i 
the universal from the parochial. 

For the past decade many "western" countries have participated 
In the country educational planning program of the Organisation for 
Economic Co-operation and Development (OECD).* Through this program, 
these Cwui\trles have shared problei&e, experience, teobnlcal assis- 
tance} they have engaged in mutual evaluation of planning activities; 
they have supported research on various dimensions of educational 
planning. The initial motivation for this cooperative activity was 
a common concern for planning the expansion of formal educational 
systems to generate more schooled manpower to support economic 
growth. With this was coupled concern for efficiency of education 
and, increasingly, for equal educational opportunity. Each of these 
elements — growth, efficiency, equity — has undergone change in inter- 
pretation as planning experience cumulated in the 1960 's. Problems 
have been reconceptualized and views of the planning process are 
changing. 



Twenty countries signed the OECD Convention in Paris, December 1960. 
They were the countries of Western Europe (members of the antecedent 
organization, the OEEC — the Organisation for European Economic Co- 
operation), plus the United States and Canada. Japan and Finland 
later acquired full membership* Australia became a full Member this 
year (1973)* All Member countries participate in the Country Educa- 
tional Planning Program; Yugoslavia also participates in It as a 
special Member. 
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In the period 1960-1970 there is evidence of significant shift- 
ing in views of the context, the agenda, the process and the time 
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perspective of planning. In brieft 



- Views of the context for educational planning are 
changing. Preoccupation with economic growth is 
giving way to a broader concern for "quality of life," 
with concomitant erosion of the growth-is-progress 
assumption in development planning generally and in 
educational planning, 

- Views of the appropriate agenda of planning are 
changing. Quasi-exclusive emphasis in planning and 
research on 'para-educational' analysis (what goes 

in and what comes out of schools, but not the educat- 
ing process) is increasingly seen as a necessary but 
not sufficient agenda. Increasing attention is being 
paid to processes of learning and Innovation. What 
education, for what purposes^ for whom, where, when, 
how? — these questions are pushing for attention along- 
side the how-much, how-many, what-cost preoccupations 
of much of the 1960'8, 

- Views of the appropriate time horizon and time per- 
spective in planning are changing. Increasing atten- 
tion is given to longer-term perspectives and to 
estimating the future consequences (negative as well 
as positive) of short-term planning. 

- Views of the planning process and ot the roles of 
professional planners are changing. The view that 
planning is a purely technical function appropriately 
performed by centralized experts is giving way to con- 
ceptions of the place of planning within the political 
decision-making process and reappraisal of the role 

of professionals in a process diffused among diverse 
publics of education. 

The shifts in views quickly sketched above derive only partly 
from evolution in planning experience. Pressures external to the 
technical planning process have been a major influence. There is 
ample evidence, for example, that conventional assumptions about 
the nature and locus of planning in relation to decision-making 
have been and are being challenged by interest groups who insist on 
being included in processes which affect their future. In the late 
1960 's that challenge came to a head in several countries, generat- 
ing crises which planning had not anticipated and was not prepared 
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to deal with* The crises were labeled; the ''relevance crisis," 
reflecting dissatisfaction with formal education in the context of 
life in the emerging world; the '^participation crisis," attesting 
dissatisfaction with decision-making processes wherein ends and 
means of education are shaped. The "relevance crisis" (education 
for what?) is, of course, intricately interwoven with the "partici- 
pation crisis" (who shall decide?), 

It has become clear in countries committed to planning that, 
if planning is to be a relevant guide to choice and action, it must 
satisfy at least three conditions* It must examine ends as well as 
means; it must be integrated in the decision-making process; and it 
must be open to participation by multiple publics in the polity of 
education . 

Such conceptions of planning have major Implications for the 
roles and styles of operation of those charged with developing the 
planning plocea^. * ^fothing'in -the r^ce^^t -wpha^^ls upon broa^ par- 
ticipation in planning denies that there are activities appropriately 
performed by a centralized group having certain specialized capa- 
bilities — at V7hatever level of government ♦ Nevertheless, if we are 
seeking to create planning societies rather than planned societies, 
then the roles of the professionals charged with planning cannot be 
purely technical, They must be redefined. To the extent that the 
professional planner becomes a broker of the planning process then 
his activities and roles and the competencies he needs require re- 
appraisal . 

D. From Statewide Testing Legislation 
to Educational Policy Planning 

We began this discussion by presenting a detailed analysis of 
a set of legislative acts in a small group of states. We leave it, 
for the present, with an emphasis upon development of the educational 
policy-planning process. 

Legislatures are a part of traditional structures of decision- 
making » Yet legislation is a policy instrument which, while 




addressing some substantive Issuei can bring about changes in tradi- 
tional decision structures^ Thus> the legislation examined deals 
with Issues of equity, efficiency and quality In education by pro- 
viding for particular evaluation procedures and assigning responsi- 
bility for them. At the same time, many of the statutes affect 
decision structures by requiring new mechanisms for planning and 
participation, 

The legislation is a valuable indicator of the salience of Issues 
here and now. But neither the basic questions at issue nor the prob- 
lems encountered in confronting them are unique to a small group of 
states that have enacted a particular kind of legislation. 

The fundamental questions at issue are universal and continuing 
rather than parochial and transient* They concern what education is 
for, whom it is for, what form it shall take, where and when and how; 
they concern how to evaluate the effectiveness of schooling in this 
context and who shall be responsible for the evaluation and for Its 
outcomes; and they concern who should make Judgmexxts about all these 
matters, on what basis, and by what means. In the complex and chang- 
ing world in which we live, we cannot afford to deal with these Issues 
in ad hoc fashion* There is a premium on devising effective policy 
planning processes in which these fundamental questions can be given 
continuing attention* 



ErJc 149 



THE QUEST FOR BETTER SCHOOLS 

Statewide Testing Legislation and Educational Policy 
by 

Maureen MacDonald Webster 



FOOTNOTES 

1- The COOPERATIVE ACCOUNTABILITY PROJECT (CAP) Is financed through 
Title V, Section 505, of the Elementary and Secondary Education 
Act of 1965 and Is administered through the State of Colorado • 
CAP Is developing a comprehensive program for servicing the 
accountability needs facing state and local education agencies. 
Among other things, It provides a central source of Information 
dealing with practices and procedures for developing and Imple- 
menting accountability and/or assessment programs. This documen- 
tation Is collected and analyzed at the State Educational Account- 
ability Repository (SEAR), located In the Wisconsin Department 
of Public Instruction. The texts of legislative acts, a primary 
source for the present paper, are recorded In CAP/SEAR publica- 
tions: Legislation by the States; Accountability and Assessment 
In Education (Denver, Colo.: Cooperative Accountability Project, 
1972), And Revised Version, April 1973. There Is also available 
now a review of accountability legislation and proposed models 
for legislation: Phyllis Hawthorne, Characteristics of and Pro -* 
VO£>ed Models for St&t^ AcccBntablllty Leg islation (Denver, Colo.: 
Cooperative Accountability Project, April 1973). 

2. A major supplementary source of Information on testing/assessment 
programs Is: Educational Testing Service and others. State 
Ed^ ^ catlonal Assessment Programs (Princeton, N.J.: Educational 
Testing Service, September 1971). [Henceforth referred to as 
*'the ETS Survey."] An updated survey of programs Is In course 

of preparation (Spring 1973) with expected publication by Fall 
1973. 

3. Two sources, setting forth features of viable accountability/ 
assessment programs are: (1) Sheila Krystal and Samuel Henrie, 
Educational Accountability and Evaluation , PREP Report No. 35 
(Washington, D.C.: U.S. Office of Education, 1972). [This source 
clarifies processes inherent in an accountability system.] 

(2) Nancy L. Bruno, Paul B. Campbell, and William H. Schabacker, 
Statewide Assessment! Methods and Concerns (Princeton, H.J.: 
Center for Statewide Educational Assessment, Educational Testing 
Service, c.1972). [This source includes an elaboration of prin- 
ciples to guide successful assessment programs.] In practice, 
the term "assessment" is also used to refer to programs which 
fall short of the characteristics set forth in these two sources. 
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4. A 1967 survey of testing programs Identifies 74 state testing 
programs In 42 states. Most of these were for student guidance. 
17 states used tests to help evaluate Instruction. 13 states 
used tests to assess student progress. See; Educational Test- 
ing Service, State Testing Programs:__A Survey of FunctlonSa, Tests 
Materials and Services (Princeton^ N.J.: Educational Testing 
Service^ 1968). 

5. ETS 1971 Survey, p. xiii. 

6. The New Jersey and Illinois participatory programs are described 
In two papers prepared for a Meeting on Participatory Planning 
In Education under the Program of Country Educational Planning 
of the Organisation for Economic Co-operation and Development, 
January 1972. (1) Bernard A. Kaplan, Developing a Participatory 
Process for Educational Planning: The New Jersey Experience 
(DAS/EID/72.41) (Paris: OECD, December~l972) . (2) Thomas A. 
Olson, The Illinois Experiment In Participatory Planning (DAS/ 
EID/72.40) (Paris: OECD, December 1972). 

7. Based on a report In Education USA , March 1973. 

8. Based upon examination of Information in the ETS 1971 Suirvey. 

9. Albert Shanker, "Below-grade Publicity About Reading Scores," 
New York Times, 25 March 1973. 



10. 


Reported 


in 


Education USA, 4 December 1972. 


11. 


Reported 


in 


Education USA, 21 September 1970. 


12. 


Reported 


in 


Education USA, 16 April 1973, p. 182. 


13. 


Reported 


in 


ECS Legislative Review, 14 May 1973, p. 4. 


14. 


Reported 


in 


The Report on Education Research, 6 December 1972. 


15. 


Reported 


in 


The Chronicle of Higher Education, 9 April 1973. 


16. 


Reported 


in 


The Report on Education Research, 9 May 1973, p. 9. 


17. 


Reported 


in 


The Report on Education Research, 1 February 1973. 


18. 


See: Suing 


the Schools for Fraud: Issues and Legal Strategies. 



Transcript of a Conference: Fraud in the Schools — co-sponsored 
by the Educational Policy Research Center at Syracuse, the Law- 
yers^ Committee for Civil Rights Under Law, and the Educational 
Staff Seminar, Washington, D.C., March 9, 1973 (Syracuse, N.Y. : 
Educational Policy Research Center, SURC, 1973). 
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19* Ibid., p,71. 

20 • Reported in The Report on Education Research , 17 January 1973, 
p. 10; 

21. National Education Association, Accountability and the Teacher ^ 
Briefing Memo (Washington, D.CO , No, 1, January 1973. 

22. Derived from a listing of resolutions given in 'fhe New York 
Teacher, 18 March 1973. 

23. Reported in The Report on Education Research , 14 February 1973. 

24. Reported in Education USA , 16 April 1973, p. 182, on the basis of 
information from Elaine Trudeau of the State-Federal Information 
Clearinghouse for Exceptional Children. 

2r. Reported in Education USA , 17 April 1972. 

26. Reported in Education USA , 15 May 1972, on the basis of informa- 
tion from Nader attorney Donald Ross. ETS agreed to give full 
cooperation in this investigation of its activities, 

27. Reported in The Report on Education Research , 22 November 1972, 

28. Henry S. Dyer, Recycling the Problems in Testing . An address 
presented at the Invitational Conference on Testing Problems, 
New York City, October 28, 1972. 

29. Wayne H. Holtzman (University of Texas), The^ Changing World of 
Mental Measurement . Presidential address of Division 5, pre- 
sented at the annual meeting of the American Psychological Associ- 
ation, Miami Beach, Florida, September 1970. 

30. See also: Maureen M. Webster, Three Approaches to Educational 
Planning (Syracuse, N.Y.: Center for Development Education, 
March 1970), Occasional Paper No, 1, 60 + xxiii pages. Pp,4-6 
discuss social influences on student achievement in the context 
of appraising the demand-for-places approach to educational 
planning; and Annex I, pp.ix-xiii> has a tabulation ^nd biblio- 
graphy on System Inputs and Process Variables Believed to be 
Related to Academic Performance (based on material prepared by 
Thomas Corcoran). A valuable review and synthesis of research 
findings on the effectiveness of schooling is available from 
RAND: Harvey A. Averch and others. How Effective is Schooling? 

A Report prepared for the President's Commission on School Finance 
(R-956-PCSF/RC) (Santa Monica, Cal. : RAND, March 1972), 
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31. The rejection of the "pool of ability'* notion and of emphasis on 
genetic determinants of ability is clear in several policy-related 
documents and in research, particularly on equal educational 
opportunity, in the U.S. and many other countries. In Europe, 

for example, it is reflected in the work of the Robbins Commission 
on Higher Education In Great Britain, and in many documents pro- 
duced through the program of the Organisation for Economic Co- 
operation and Development (OECD) from the 1961 KungSlv Conference 
on Ability and Educational Opportunity onwards, 

32. Torsten Husen,ed,, International Study of Achievement in Mathe - 
matics (Stockholm: Almquist & Wiksell; New York: John Wiley & 
Sons, 1967). International Project for Evaluation of Educational 
Achievement, Hamburg. Vol, 1 explains purposes, procedures, test- 
ing instruments and administration of the research in 12 countries. 
Vol* 2 presents results of the mathematics achievement study, 
recounts problems and limitations, and outlines plans for the 
next phase of the evaluation project. For a brief account of 
subsequent research, see Douglas Pidgeon, "Current Research of 

the lEA," Comparative Education Review Vol. 13 No* 2 (1969): 213-216. 

33. Many of the large-scale analyses use performance on norm-referenced 
tests as the index of student achievement — encountering the prob- 
lems of interpretation and psychometric properties of these tests. 
This further underscores the ambivalence of the messages relayed 

by contemporary research to the world of decision-making. 

34. Harvey A. Averch and others, How Effective Is Schooling? A 
Critical Review and Synthesis of Research Findings (R-956-PCSF/RC) , 
(Santa Monica, Cal.J RAND, March 1972), p, 48, 

35. This is a finding from analysis of data in the report by J. S. 
Coleman and others, Equality of Educational Opportunity (Washing- 
ton, D.C.: National Center for Educational Statistics, USGPO, 
1966). 

36. Educational Quality Assessment, Phase II Findings. Data Analysis . 
(Harrisburg, Pa.: Pennsylvania State Department of Education, 
1971), p, 34. 

37. A 1971 survey, funded by USOE, was completed by Missouri State 
Department of Education, Planning and Evaluation Section — The 
Status of Comprehensive Planning and Application of System Analysis 
Concepts in Planning by State Departments of Education for F.Y. 1971 . 
Jefferson City, Missouri, December 1971. 114 pages. Processed. 
Another survey of planning is being completed in 1973 by the New 
Jersey State Department of Education (Research, Planning, and 
Evaluation Division). Several useful additional documents are 
available, many of them from projects funded under Title V of ESEA. 
These include sets of documents from the Institute for State Edu- 
cation Agency Planners; the Seven State Project (B. S. Furse and 

L. 0. Wright, eds.. Comprehensive Planning in State Education 
Agencies ) ; the Eight State Project ( Designing Education for the 
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Future series) } Improving State Leadership In Education Project, 
and others* 

In-depth analysis and documentation of these shifts In views of 
planning Is presented In Part One> "Educational Planning In the 
Sixties/* of Maureen M» Webster^ Educational Planning In Transi - 
tion — Emerging Concerns and the Alternative Futures Perspective 
(Syracuse, N,Y.: Educational Policy Research Center, SURC, 
August 1971), For a summary of the major trends, see Maureen 
M. Webster I "Planning Educational Futures — Some Basic Questions » 
Journal of Educational Planning (Winter 1972). 



Chapter Three 



WHAT ARE SCHOOLS FOR? 

The Issue of Minimal Skills 



by 

Naomi Rosh White 



PART ONE 



INTRODUCTION AND OVERVIEW 



Within the past five years, increasing legislative attention has 
been directed to the issue of accountability. "Accountability" has 
been variously interpreted to apply to teacher evaluation, some form 
of systems accounting, or student evaluation as the means for ensur- 
ing that schools are meeting their responsibilities. This paper 
addresses some of the educational issues which impinge upon one form 
of accountability legislation — that mandating statewide testing as 
the m^ans for assessing the "adequacy and efficiency" of schools. 

The statewide testing which is mandated in legislation, and that 
which is occurring without legislation surfaces many complex patterns 
of operationalization . Conanon to these diverse patterns is an empha- 
sis on minimal skills as the basis for assessing the adequacy and 
efficiency of schools. The movement to statewide testing as an 
accountability tool, however, begs many of the controversial ques- 
tions underlying the use of particular evaluation tools (standardized 
tests) as measures for achievement in the minimal skills (most fre- 
quently conceived as reading, writing, and arithmetic). 

The 3R^s represent only one way of understanding minimal skills. 
The purpose of this paper is to demonstrate that there are alter- 
native conceptualizations of basic skills which have consequences 

k 

See Table 2.7, p. 78: "Operational Goals; Minimum Requirements 
Specified in Testing/Assessment Legislation." Appendix D: Educa- 
tional Testing Service: Summary of the Statewide Testing Programs — 
mandated and non-mandated — which documents the current emphasis on 
the 3R 3 as the focus for evaluation. 
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for 1) the basis for identifying educational goals; 2) the viability 
of testing as a means of evaluating educational performance; 3) the 
agencies which can legitimately be held responsible for Identifying 
and Insuring possession of certain skills* 

Two Meanings of '^Minimal Skills" 

Broadly speaking, minimal skills can be said to have two mean- 
ings. These meanings are not mutually exclusive, but represent dif- 
ferences In emphasis. 

^^Mlnimal" as "Prerequisite for Learning" 

This' usage Is predicated ou an ordinal or hierarchical Station 
of learning. That is, the student, in order to come to know certain 
things, must possess basic skills which will facilitate acquisition 
of further knowledge. These skills are necessary antecedents to 
cognitive activity, antecedents both temporally and logically. 

It would seem to follow, then, that the appropriate perspective 
for exploring this interpretation of minimal skills would be cogni- 
tive and learning theories. These theories might provide evidence 
for what basic skills are, how one acquires them, and how one might 
test for them. 

"Minimal" as "Necessary for Personal Efficacy" 

This usage is predicated on the view that everyday existence 
requires of each person skills necessary for survival. That is, a 
person, in order to function effectively in the social environment, 
must possess certain skills. It further implies that one Is fully a 
person only if one possesses these skills. The skills are necessary 
and sufficient for — personhood — self-actualization* . .whatever terms 
one cares to choose. This stands in marked contrast to the usage 

ERIC 



described above, where the skills are necessary for learnlngi but not 
sufficient* 

There Is no necessary allusion to a temporal or ordinal sequence 
associated with the acquisition of '^minimal skills'* In this second 
sense. In practice, however, the acquisition of some minimal skills 
in the first sense might be logically and temporally prior to "mini- 
mal skills'' in the second sense* The appropriate perspective for de- 
termining minimal skills for this meaning is social theory, as social 
theory helps describe the conte;xt for which these skills are intended. 
For the moment, it should be noted that there is considerable am- 
biguity as to the parameters or boundaries of this context, and there- 
fore, the nature of the skills deemed to be sufficient. Are skills 
intended for survival In tlio concrete Jungle of the inner city? Are 
they intended for survival both in the inner city and a different 
social or cultural milieu » namely, a predominantly middle-class 
environment? 
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Sometimes both meanings are ascribed to one set of minimal skills. 
In other words, one hears that the 3R's are necessary antecedents 
for further learning. One also hears that the 3R's are necessary for 
one to be a good citizen. One must be able to read signs, give the 
correct change and fill In taxation forms. But clearly, the latter 
represents a very limited view of the "good citizen." Both senses 
of "minimal skills" espouse a particular view of man, society and 
learnings The first sense does so implicitly, the second explicitly. 

What does all this have to do with educational policy? Policy- 
makers articulate a concern to make the schools accountable for stu- 
dent learning outcomes. One way they operatlonallze this concern is 
through legislation for statewide testing of minimal skills. Ex- 
amination of this legislation raises questions as to the grounds on 
which 1) particular conceptions of basic skills were formulated, 
2) the decision was made as to how the possession of these skills 
should be evaluated, and 3) statewide^ miform (v. locally determined, 
possibly diverse) skills were decided upon as appropriate. 
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In other vords, In what sense are the skills basic? What are the 
justifications for this view? Why evaluate them in this way? The 
answers to these questions, as manifested in the legislation, have pro- 
found implications for who succeeds at school, and why they succeed. 
In other words, the means of dealing with learning outcomes carry 
with them implicit resolution of pedagogical and social issues. I 
plan to examine the implications of the statewide testing legislation 
with these concerns in mind. 
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PART TWO 

MINIMAL SKILLS IN THE CONTEXT OF EDUCATIONAL THEORY I 



SOME ALTERNATIVES 



This section describes the approaches of three cognitive theo- 
rists whose work directly addresses the issue of minimal skills. The 
first two I Gagne and Plaget, offer conceptualizations of minimal 
skills which are empirically validated alternatives to the notions 
espoused by some of the current legislation. Montessorl, whose work 
will also briefly be described > addresses skills which she claims are 
prerequisites for even the minimal skills of reading, writing and 
arithmetic* 



Gagn4 

Gagne I the first to be considered » is a prominent learning theo- 
rist.^ His approach requires appraisal of each task to be performed 
in terms of hierarchies of prerequisite concepts and skills. If 
learning at any level is to occur with the greatest facility, careful 
attention must be paid to the prerequisites of such learning. The 
Implications for what one considers ^'minitrxl skills" are quite clear. 

It will be difficult for the child to leatn the de- 
finitions (principles) of geometry unless he has pre- 
viously acquired the concepts of line, angle, triangle, 
intersection, and so on* It wilJ. be difficult for a 
learner to acquire the principles of any specific 
science unless he already knows some more basic prin- 
ciples of classifying, measuring, and Inferring. It 
is demonstrably difficult for a learner to construct 
meaningful utterances in a foreign language unless 
he had learned the concept words that compose such 
coaumjni cat ions i and it is difficult for him to learn 
these words unless he has previously learned to say 
the sounds of the language^ Learning to read English 
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comes hard to those who have not first learned to 
speak many English words »2 

It cannot be said that any of these ''shortcut'' kinds of learn- 
ing are Impossible. Nevertheless, Gagne claims that shortcuts carry 
their own handicaps, and typically result In deficiencies that show 
themselves as limitations In generallzablllty of the capabilities 
acquired • 

* ^'The fmdcmentals of inatruotion are not clearly conveyed by 
euoh expreeeione as 'reading^ writing^ and arithmetio ^ . There are 
capabilities which cut across this formulation of basic skills. Two 
points emerge here. The first Is that there are skills more "mini- 
mal'' than the 3R's; the second, and more Important point, is that 
there are ways of describing the skills necessary for any given task 
which may or may not place the 3R's within an array of important 
prerequisite skills. In subjects like mathematics and science, the 
most basic capabilities are to be found in the stimulus-response 
connections, chains, and concepts that make up the activities of 
observing, discriminating,, drawing, and classifying^ to name a few. 
They also are to be found in the activities which constitute problems- 
solving or critical thinking, minimal skills which ;3tand apart from 
the 3R's. 



Pla ge t 

Piaget writes about the stages of cognitive development.^ These 
stages constrain the sorts of learning activities the child may fruit- 
fully engage in» Or, more correctly, the child may engage in nearly 
any activity, but the learning may differ markedly from what the adult 
would expect. Piaget 's conception of cognition is of an ordinally 
emergent set of capacities or processes of thinking which vary quali- 
tatively. This conception has consequences for both notions of minimal 
skills outlined earlier. For example, Piaget would claim that the 
child's language and thought are different from the adult's. He 
would also claim that the character of demands made by intellectual 
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tasks In adulthood Is discontinuous with the character of demands In 
childhood.^ 

Plaget claims that chronological age Is not a reliable Indicator 
of cognitive stage. This has consequences for minimal skills con^ 
celved as prerequisites. Although stages are roughly equivalent to 
chronological age groupings » there are considerable variations within 
these groupings* So» at a given age leveli children's cognitive 
structures differ. Therefore) the things which they can learn at a 
particular age level differ. Plaget recasts traditional conceptions 
of "minimal skills*' to refer to thought processes rather than specif** 
ic skills such as reading* Tho qualitative changes occurring with 
the development of thought processes » plus the chronological cor*^ 
relates of various stages of development speak directly to a re- 
formulation of the goals of instruction — that isi to what one shall 
evaluate and the methods or tools to be used. 

Plaget *s work has implications for the use o£ standardized tests 
to measure the acquisition of any skills » tests which are referenced 
per chronological age group. Ginsburg and Opper write that Piaget^s 
clinical method has shown that the child's Initial verbal response 
(the type of response given to a standard test) is often superficial 
and does not provide a reliable index of the real quality of his 
understanding.^ Moreover, the method of evaluation utilized by Pla- 
get involves intensive one-to-one interviews which span a number of 
hours. This stands in marked contrast to mass evaluation techniques. 
Tests often tap only the surface, Plaget would argue, and they off^n 
test the wrong things. Questions of economy play a role in the de- 
termination of adequate evaluation tools. Plaget' s method is time- 
consuming and costly. But a dilemma remains. How does one "price" 
the consequences of misused standardized tests? 



Mpntessori 

Montessori identifies three stages preparatory for academic 
learning. These are: motor education, sensory education, and 
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language education. By four years of agei children are typically 

ready to engage In activities pertinent to reading > writing and 

arithmetic. Thd latter are considered to constitute ''academic'^ 

learning, Solvelga MlezltlSi In her paper on the Montessorl method i 

writes that presumably the child who has mastered the earlier stages 

la adec.uately coordinated » capable of ordered observations of the 

environment and sufficiently capable of the Initiative^ responsibility 

and cooperation required to approach the prescribed sequences ln~ 

7 

volved in learning academic skills. She continues to say that a 
meaningful evaluation of the effects of the Montessorl method on cog- 
nitive functioning should Iriciude measures of cognitive style char- 
acteristics, in addition to general measures of intellectual func- 
tioning and more specific measures of achievement in the areas of 
perceptual, conceptual and language development. Hiezitid is there- 
fore making an argument for the assessment of characteristics which 
go beyond those of reading, writing and computational attainment* 

There are many other views which could be presented. These three 
should suffice to indicate a range of alternatives to the 3R*8 con- 
ception of minimal skills which have consequences for what is taught 
and how it may be evaluated. In other words, the modes of evaluation 
chosen should be consistent with the theoretical underpinnings of 
the cluster of skills to be evaluated. Further, how one defines that 
cluster may vary with each theoretical system, and has consequences 
for what it is the schools are being held accountable. 
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PART THREE 

TESTING AND ACCOUNTABILITY < SOME PROBLEMS AND ISSUES 



The "Critical Age'^ Problem 

The three theorists cited above Indicate when and in which order 
the skills they consider basic may be acquired* The statewide testing 
legislation^ In mandating tests In certain yeard> by default pre- 
scribes when these skills should be taught and attained^ Mandating 
reading tests for accountability as early, In soue states » as grade 1 
(e«g,, California) and In others » grade 3 (e.g*» Arizona) then raises 
the further question as to the likelihood of the curriculum and In- 
struction being modified to meet the outcomes sought by the tests » 
regardless of whether these outcomes are compatible with the best In- 
terests or current capabilities of the students. The effect of 
tests on what is taught in schools was demonstrated with the perceived 
redirection of high school curricula to Improve students' College 
Boards examination results. The consequences of prescribing systematic 
testing programs » especially those which start in the early years of 
schoolings need to be carefully evaluated before such procedures are 
locked into place in legislation. 

The practice of uniform testing appears to rest both on a lack 
of knowledge of the consequences of its inception^ and on social 
and cognitive theoretical assumptions for which definitive empirical 

it 

Age and grade level are tightly correlated. Grade 1 age level is » 
on average, 6 years » for instance; grade 3, 8 years. The deter- 
mination of students* best interests or current capabilities per 
age or stagi depends on the theory of development espoused and 
the social perspective from which the student and school are 
viewed'. In other words, developmental and school responsibility 
issues are being resolved without their being explicated. 
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verification has yet to be established* One of these assumptions has 
been explored by Rowher in his article "Prime Time for Education: 
Early Childhood or Adolescence." He writer about the timing of de- 
mands made on students for learning particular contents and skills 
during specified periods of schooling* These demands may be Justi-- 
fled by the notion of critical period (or age), as in the case of 
reading or language acquisition, or in terms of the presumption that 
the skill or content is prerequisite for some subsequent learning. 
Either means of Justification is defensible provided that it lA de- 
monstrably valid. The problem is that for many kinds of school learning, 
neither a critical period nor prerequisite status has b een demon* 
strated empirically.^^ 

The ambiguity in the notion of "critical age" has important con- 
sequences both for testing and the conceptions of schooling and mini- 
mal skills adopted. Reading instruction, for example, generally be^ 
gins with the onset of formal schooling, usually at age six, and the 
child's progress in reading typically becomes the major criterion for 
Judging both his success at school and the effectiveness of the 
school. One Justification commonly articulated by proponents for 
teaching and testing the 3R's in the early grades rests on research 
evidence which suggests that the Inability to read by grade three 
correlates highly with failure in the later years of schooling. The 
conclusion .'lommonly drawn from this evidence is that third grade, 
or age eight is the critical age for the acquisition of reading, for 
instance. But this conclusion may not necessarily follow. 

"Critical age," as indicated earlier, can mean two things. The 
first use refers to a predisposition or readiness on the part of the 
student which reaches its optimum at a particular chronological age. 
The implication here is that unless advantage is taken of this age, 
acquisition of the skill in question becomes difficult if not im- 
possible. The "predispositional" sense of "critical age" repre- 
sents a direct appeal to cognitive developmental theory. As indi** 
cated in the discussio;^ of Piaget, Gagne and Montessori, there are 
within this domain varying conceptions of both "minimal skills" and 
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the potency of claims as to ^'critical agei** None of the theories can 

be taken to be definitive, though each has generated substantial re- 

p 

search and empirical verification* There is no compelling evidence 
that delaying the onset of reading instruction by one or several years 
would rotard the rate at which skills are acquiredt^^ 

The second meaning which may be attributed to "critical age" 
relates not to characteristics of the student, but to structural char- 
acteristics of schooling, tt is contingent upon a particular way of 
organizing learning and teaching. In other words, the second inter- 
pretation follows from the fact that after grade three, teaching be- 
comes increasingly reliant on books, written reports and written 
tests, causing the inability to read and write to constitute a dis- 
advantage. It then follows that "Age 8 is the 'critical age^ for 
acquiring the 3R*s," Given this interpretation, it is not surprising 
that research provides evidence to substantiate the claim that in- 
ability to read or write precludes success in higher grades. 

The relevance of this distinction for educational policy is that 
the "predispositional" notion of "critical age" is not directly sub- 
ject to policy decisions. It is a "given" — a characteristic of the 
population to be served. One cannot change the age at which predis- 
positions occur for learning certain skills. The predispositional 
aspect of "critical age" is amenable to policy intervention only in- 
sofar as different theoretical perspectives result in differing pe- 
dagogical practices (so that a Montessori school might look different 
from a school based on Plagetian principles), and so on,* This then 
leads one to the second notion of "critical age," but it is not ne- 
cessarily related to it. 

The "prerequisite" notion of "critical age," on the other hand, 
is within the realm of educational policy, as it relates to decisions 
about the structure and substance of nchoollng. One could conceiv- 
ably have a school structured so that there were no Heeeaaary 

— ■ - - — 

It would also follow that different evaluation procedures would 
also be appropriate or necessary. 
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prerequisites » Such a school could derive Its organization from 
assumptions about the non-def Inability of the structure of knowledge, 
assumptions which are Independent of developmental notions. Depend- 
ing on the meaning of ''critical age*' one adopts, alternative policy 
strategies emerge. One of these Is briefly described below. 

There Is no persuasive evidence that reading Is the only means 
by which the student can acquire the other kinds of Information that 
might be useful to learning during the first five years of schooling. 
Those Involved with visual literacy would dispute the notion that the 
stimuli for cognitive activity are solely to be found In printed 
words. They point to an Increased Importance of film and television 
as educative devices. These devices could be used to foster non- 
literary communication and other skills whose attainment, depending 
on the conception of "minimal skills" one espoused, might be seen 
as desirable and appropriate. 

In other words, 

(a) If the notion of minimal skills as prerequisites to 
further learning is no longer so tightly tied to current practices 
of schooling (which includes the dominant conception of the 3r's 
as minimal skills), and 

(b) if one accepts the utility of skills such as reading, 
writing and computation, and 

(c) given the pressing concern foi: information about the 
outcomes r^t schooling, 

one resolution might be to prescribe the assessment of these skills 
in the last two or three years of compulsory schooling. This would 
both allow for remedial treatment of those students who may have 
moved through the alternative system without acquiring certain 
skills and enable the school to fulfill its responsibility to these 
students. Assessment of whether the schools were "doing their Job" 
would be tied to some positive compensatory means by which any 
shortcomings of the school would be directed at those for whom the 



skills were deemed necessary • This would not preclude, and In fact 
might conceivably serve, to encourage the development or learning 
of other skills during these years* 



Evidence exists to assuage the concern of those who may well 
say "I want the school to teach my children the 3R's,'' It is possible 
to teach these skills in later years. Examples of motivated teenagers 
and adults who have learned to read and write with intensive instruc- 
tional programs lasting for six months have been cited in educational 
literature,* There is still, of course, the unanswered question of 
the nonschool-relaLed costs for people who do not learn to read in 
the earlier years « 

What if concern is expressed by parents about the lack of ex- 
plicit attention to the 3R's in the earlier grades? Perhaps another 
means might be utilized for making available to parents and relevant 
agencies information relating to students* activities. This informa- 
tion might come in the form of teacher reports, in which the 3R's — or 
activities related to them — may or may not be included in the range 
of alternative activities for which student participation is required 
or demonstrated. Such reports would preserve a flexibility of cur- 
riculum and pedagogy while satisfying the responsibility for public 
accounting. If objectivity of reporting is a concern, procedures for 
validating teacher reports could be incorporated into the system. 
For instance, parents might be invited to participate in the evalua- 
tioni Local school district (and even state) personnel could be in- 
volved, as they are also being held accountable* This method does 
not lend itself to easy aggregation of data as is demanded by the 
accountability legislation, but it may have educationatly beneficial 



ERLC 



Paulo Freire in Pedagogy of the Oppressed has Written of his experi- 
ences In Brazil I where he taught adult peasants to read and write 
within a very short space of time. George Dennlson, In Lives of 
Children, recounts his success at teaching inner-city teenagers 
in New York City to read and write in a similarly short time, Note 
that both these men were operating in non-traditional educational 
structures. Dennlson was operating from a store-front school, 
Freire in the homes of the peasants. 
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consequences whose priority might well be raised for discussions 
For example, changing the evaluation procedures may enable experimen- 
tation with alternative conceptions of schooling based on syntheses 
of cognitive and social concerns* Furthermore, parents could partic- 
ipate In the planning and evaluation of schooling. This participa- 
tion could perhaps be viewed as a form of adult education whose aim 
It Is to solicit suggestions and to Integrate parental demands for 
accountability Into the process of rendering It. Hopefullyi It would 
require relating the outcomes for which schools are being held account- 
able to the processes by which the schools ensure that the responsi- 
bility Is being met. 

The scenario briefly sketched above raises the following ques- 
tions which ought to be addressed: 

1) When are uniform Standards necessary and beneficial, 
and when are they dysfunctional? 

2) What is the balance to be achieved by schools between 
responsibility to the student and responsibility to 
the general public and parents? 

3) What are the domains of student autonomy 

in decision-making relative to those of the teaching 
professionals and administrators? 

These are difficult questions to answer. The scheme outlined above 
is one way in which reasonable boundaries for demands for performance 
imposed by agencies other than the individual teachers, schools, or 
school districts could be reconciled with the interests of the stu- 
dents « 

The Instrument Problem 

This section of the paper will focus on the implications of 
uniform statewide testing for the notion of basic skills one espouses 
with particular attention to the uses and abuses of testing. 

Norm-Referenced Tests 



The evaluation tools most frequently used for basic skills are 



norm-referenced achlevetnent tests. Norm-referenced tests show how 
students rank , following exposure to content, regarding the abilities 
reflected In the tests. The problems with these tests have been elab- 
orated by many, at great length. 

The most frequently cited problem Is that the tests are culturally 
biased, favoring those children from white, middle-class backgrounds, 
and discriminating against others. In othdr words, the speaker of a 
non-standard dialect has special problems when taking standard- 
ized tests* Further, these special problems cause differential 
test performances by children of different ages and different lin- 
guistic backgrounds. Elsa Roberts outlines some of the areas of po- 
tential difficulty as being (1) the content of the test questions and 

expected responses, (2) the verbal style required by the test, and 

13 

(3) the non-linguistic factors Inherent in the testing situation* 
Substantive biases in tests can include specific vocabulary Items, 
culture specific photos used in vocabulary tests, as well as culture 
specific information questions. Vocabulary tests can work against 
children of a particular subgroup in that the object which the test 
word signifies, e»g., toboggan , can be outside the experience of such 
children, or the word, e.g., spectacles , Itself can be different in 
the language system of the subgroup. By the same token, certain 
information questions can work against certain children. 
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The cultural bias of tests, with their resultant deselection of 
a portion of the population, focuses attention on a problem Inherent 
in the use of test scores for accountability. Norm-referenced tests 
contain an ambiguity of purpose which is encapsulated in the term 
"achievement.'' As mentioned above, the tests are in part designed to 
predict future academic success. Future academic success in fact 
constitutes one of the validating criteria for such tests. "Achieve- 
ment" here refers to student potential for achievement. On the other 
hand, tests rank students according to test scores, so that "achieve- 
ment" might refer to a quantified statement of a student^ s achieve- 
ment and relative status at a given point in time. "Achievement" 
here refers to a product . Future success and current achievement 
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correlate with a range of factors* Some of these fall within the do- 
main of the school* s sphere of interventive action--«such as teaching 
method^ classroom environment and so on$ others » such as socio-econo- 
mic factors, clearly do not* Accountability legislation, with its 
interest In outoomest is effectively concerned with "achievement'^ as 
attainment at a given point in time* (That is, none of the legislation 
requires follow-up studies to determine the post-secondary success 
or fate of its students*) The appropriateness of the norm-referenced 
standardized test for providing this information rests on assumptions 
which are problematic. The broader issue relates to the grounds on 
which Judgments are made regarding the standards which all students 
ought to achieve, and| consequently, for which the school ought to be 
accountable. The particular problem arises when the te^ta come to 
eerve as the standards for that which ought to be achieved* The 
phenomenon of tests taking on the function of standards is a consequence 
of accountability legislation which assesses "the adequacy and ef- 
ficiency of schools" in terms of the information the test provides* 
This represents a patent misuse of instruments not designed for this 
purpose* 

« 

If the concern for accountability stems from a concern for an 
equally distributed achievement of specified standards — a product — 
norm-referenced tests are substantively inadequate. Examination of 
the test items would yield few attempts to Justify the view that the 
substance of these tests provides a desirable standard for student 
outcomes. This is hardly surprising^ considering the intent of the 
instruments* Test items are samples of a range of possible items i 
chosen in part with a concern for content validity, but also deter- 
mined by "statistical" criteria* That is, given content validity, does 
the test item differentiate between students, and between levels of 
difficulty per test item? The statistical (versus substantive) cri- 
terion is critical because the norm-referenced test is intended to 
differentially rank students. Thus, statistical considerations re- 
sult in the inclusion of some items and the exclusion of others* 

The choice of items for inclusion in the test relates the issue 
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of standards to that of the cultural bias of tests. Whose criteria 
for standards regarding substance ought to be utilized for assessing 
the ''adequacy and efficiency'* of schools? In other words, if the 
tool one uses to assess '^the adequacy and efficiency of schools'* con- 
sistently discriminates against a portion of the population due to 
criteria which in themselves are problema^^c, i$ one not unwittingly 
maklvig discrimination a criterion for "aiesiuacy''? It is important to 
not^^ that the argument being made is not an argument against standards. 
It relates to the grounds on which one may arrive at such standards, 
the purposes or goals these standards serve^ and the mode of eval- 
uation adopted. 

What if the problem of identifying standards for communication 
skills^ for Instance, was resolved? Could norm-referenced tests be 
used for diagnostic purposes^ e.g., one might link the evaluation 
very closely to instructional programs specifically attending to lan- 
guage difficulties. In this event, norm-referenced tests would be 
less appropriate evaluation tools than criterion-referenced tests. 
Norm-referenced test results can provide only a comparative ranking 
of achievement scores. Mastery ^ or criterion-referenced tests are 
more suitable tools for the assessment of instruction. One mayi and 
ought to, teach directly to a criterion-referenced test; this prac- 
tice is clearly not appropriate for"* norm-referenced tests. Norm- 
referenced testing cannot directly address the question of instruc- 
tional effectiveness, or of the "adequacy and effectiveness^' of 
schools. That is, the tests, especially as they are being used for 
accountability, beg the question of school (versus other environ- 
mental factors) Influence on achievement, relative to aptitude, and 
therefore, the extent to which schools can be held accowtable for 
student achievement • 



Norm-referenced tests are designed to Indicate a spread of achieve- 
ment or aptitudes. The tests are constructed so that the items are 
differentiated according to level of difficulty, and the resulting 



These are described in greater detail on pages 175-176. 
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spread of scores is supposed to approximate a normal curve. The 
utility of these types of test results for accountability is further 
circumacribed. The emphasis on educational outcomes in the account- 
ability legislation is not an attempt to maintain a spread of scores 
and J therefore f of achievement. The concern is to identify areas 
within which students achieve at specified levels. There is a cet*- 
tain ambiguity in the legislation, which leads to one of the curious 
paradoxes inherent in the use of norm-referenced tests for account- 
ability. The paradox lies in the demand for student scores to be 
raised to the general norm.^^ If the child shows development that 
equals or exceeds the norm, it is assxmed that he is learning satis- 
factorily. If he ranks below the norm, however, as half the students 
do, concern is expressed. But, to say that half of the students fall 
below their grade level in reading, for example, is merely to state 
a statistical constant. Because the grade level in reality is 
nothing but the norm for all students of an age who have been tested, 
it is inevitable that half will rank below that norm and half above. 
To get all students above the norm scores is a statistical and log- 
ical impossibility. 

Prior experience with tests and te^^t anxiety, also have to be re- 
cognized as factors affecting the reliability and validity of test 
scores. There are statistical procedures for correction of error and 
modifications In content to decrease bias; but these corrections do 
not eradicate the problems listed above. Moreover, knowledge of test 
results has been shown to modify teacher expectations of student per- 
formance, leading to adjustments in teaching to parallel the child's 
supposed ability (as documented in the studies on self-fulfilling 
prophecies and achievement). Test scores cannot be viewed as hard 
data gathered by instruments with absolute precision and accuracy. 
The scores at best are approximations of a spread of performances on 
skills whose identification beyond the actual test item is problem- 
atic. Test scores, when sensitively used, can yield information to 
guide teaching practice. A ^'wholesale approach^' to testing, however, 
ignores social costs which should be taken into consideration. 
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The moat pressing question is, what do the tests really measure? 
What does reading at the third or ninth grade level signify? As noted 
earlier » Piaget^s work indicated that all one could expect of test 
results and responses was that they would yield superficial informa- 
tion« Test constructors themselves will often only go so far as to 
say "tests measure what tests measure," indicating that construct val- 
idity is an unresolved issue. This suggests a further question which 
requires stepping back from the tool to ask under what Qonditione 
should the pavtioular skills measured by the tests be ut^lliaed for 
acQountability . It i$ to this question that the outline of alterna- 
tive notions of basic skills is addressed. 



Criterion-Referenced Achievement Tests 

Criterion-referenced tests focus on what an individual has 
learned I not on hoW he stands in comparison to other testees. Cri- 
terion-referenced tests claim to measure an absolute level of achieve- 
ment over a specified content. The function of the test is to in- 
dicate modifications in instruction which might lead the student to 
reach the level of mastery required by the program objectives. It 
is this emphasis on program effectiveness which makes this test an 
accountability tool* 

An example of a commonly employed criterion-referenced test is 
a driving test. The component skills for driving are listed— backing, 
parking and stopping at red lights, to name a few— and mastery of 
each of these must be demonstrated by testee in order for a li- 
cense to be granted. Driving is a relatively finite skill with 
identifiable components. Social studies or science, on the other 
hand, are less amenable to such compartmentalization. 
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Criterion-referenced tests are increasingly being used for ac- 
countability purposes (e.g , Hawaii, Florida). But they carry with 
them assumptions as to the boundaries of subject matter, and concep- 
tions of knowledge as "product" rather than "process," which are nec- 
essarily incomplete or inaccurate. For example, how would one con- 
struct a criterion- referenced test to adequately indicate mastery of 
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"citizenship," or social studies? The former constitutes a domain 
which may not be amenable to "testing" as currently conceived} the 
definition of the realm of the latter represents a major Intellectual 
problem covering Issues of methodology and history at the very least* 
How would controversy as to the adequate boundaries and component 
parts for the areas covered by these tests be resolved? Who should or 
could decide these things? The uncritical use of criterion-referenced 
tests may result In fragmentation of educational outcomes on largely 
arbitrary criteria. This has Implications for the structure of the 
curriculum and for pedagogy— Implications for what schools do* The 
question which ought to be addressed Is; for what domains and under 
what conditions do criterion-referenced tests provide Information 
directly relevant to accountability concerns? 

Models for measuring differences among competing programs using 
criterion-referenced measures have not yet been developed* Compar- 
ability of| and consensus about objectives (especially In vaguely 
defined domains such as In social studies) and validity of Items 
testing for these objectives are problems central to establishing 
content validity for crlterlon^-refercnced tests, and have conse- 
quences for their use as accountability tools* The procedures used 
for norm-referenced tests are dysfunctional for the criterion tests 
because of their differing purposes. Procedures for standardising 
criterion-referenced tests have not yet been established. If local 
determination of objectives and evaluation procedures comes Into 
effect I and criterion-referenced tests are used, this methodological 
lacuna will make Inter-dlstrlct comparisons rather difficult* 

The Issues relating to the use of criterion-referenced tests 
require consideration of the second sense of minimal skills* The 
"survival" or "social" conception of minimal skills provides a,lter- 
natlve frames of reference for determining the boundaries and com- 
ponents of that which might be tested. 
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PART FOUR 

"MINIMAL SKILLS" FROM A SOCIAL PERSPECTIVE 



When one addresses the Issue of "minimal skills" from a social 
perspective, one is raising three questions. First, what is the na- 
ture of the society for which these skills are intended? Second, 
what are the "minimal skills" necessary for effective functioning in 
this society? Third, which. If any, of these "minimal skills" is it 
the responsibility of the school to foster? With the attempt to ans- 
wer each of these questions, insofar as an "answer" can be provided, 
a plathora of problems pertaining to relevant parameters of inves- 
tigation arises. Each attempt must, for example, contain particular 
ideological biases or normative stances, for that is the character 
of interpretation. Frequently, the descriptions of "how things are" 
derive from assumptions as to "how things ought to be." Such descrip- 
tions are contentious and open to debate. The range of assumptions 
about the degree to which definitive agreement on, or verification of 
"answers" is necessary or possible may have contributed to the less 
frequent attention to these concerns which have fundamental import 
for education. The difficulty of resolving the methodological Issues 
may have constituted another hindrance. Most likely, however, is 
the fact that these concerns are politically explosive, and con- 
sequently are set aside for "specialists" rather than public debate. 

That is not to say that similar problems, relating to choice 
of the operative developmental theory on which to base schooling, do 
not arise. They do, but they are more effectively draped in the 
robes of scientific expertise, blurring the issues of ideology which 
surface with any discussion of the social purpose of "minimal 
skills." It is my contention, however, that the developmental 
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theoriea provide U8 with a building structurej the structure is 
necessary » but not sufficient. The social issues turn the steel 
skeleton into a school-ln-context, with a decor and style which 
give the school its identity—set in a particular time and place. 
Consideration of these social Isauesi complicated though they may be, 
is necessavy for any adequate examination of the goals, functions and 
limits of schooling — for determining those skills for which the school 
is to be held accountable. 



The Society For Which the ^'Minimal Skills^' Are Intended 

The choice of dimensions for an account of the times in which we 
live is problematic, to say the least. It is my purpose simply to 
surface for discussion some of the pervasive themes which may be said 
to relate to the issue of "minimal skills," and ultimately to the pur- 
poses, goals or functions of schooling. These themes address domi** 
nating Ideologies and structural characteristics of the society, and 
have consequences for how one conceives the domains for which the 
individual and the school have responsibility. 

It has been said that we live in a time of urbanization, bureau* 
cratizati^««, and rampant technology. Traditional ties such as the 
family are weakening, as are the customary restraints and sanctions 
for behavior which previously were informally enforced. Formal codi- 
fication of norms by laws, together with the flourishing of bureau- 
cracies which facilitate centralized decision-making (for the masses 
by elites) fosters a sense of apathy deriving from Impotence. The 
market paradigm encourages the development of competitive, individ- 
ualistic priorities Jn the face of disintegrating traditional sources 
of shared meanings and ideals. The result is rule by technique, a 
preponderance of concern with means, rather than the ends which these 
means are to serve. The picture which emerges i» of a fragmented 
society characterized by chaotic dissensus. The repeated assertions 
of dissensus are of critical Importance to both the politics of 
decisions-making for education and to the substantive and organiza- 
tional issues pertaining to schools. These claims about dissensus 
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encotnpass thf; questions as to whether schools are simply preparatory 
training ceriters for the labor market} whether they are Institution- 
alized means of keeping youth out of the labor market j whether they 
are the means by which a society 'Is dominant beliefs and culture are 
transmitted} or whether schools are the vehicles for oppression and 
cultural 'lominatlon. These variations in ascription of the functions 
of schooling rest on a diversity of beliefs and priorities » a plu- 
ralism which characterizes the United States today. 

Vhat does one mean by pluralism? Thomas Green makes a distinc- 
tion between "pluralism'^ aa an ideal or a particular constellation 
of values I and ^'pluralism'' as a social reality made manifest in the 
social structure,^^ Utilizing Cooley*8 distinction between primary 
and secondary associations or relations, he constructs three models 
of pluralism. These models are "Insular pluralism," "half-way plu- 
ralism," and "structural assimilation." They describe a continuum 
from societal fragmentation or "pluralism," to societal integration 
or "homogeneity.'^ 

" Insular pluralism" is characterized by the maintenance of both 
primary and secondary associations within one group. The Amish are 
an example o£ such an Insular community. 

" Half-way pluralism" is the type in which primary associations 
are determined by relevant in -group characteristics, such as religion 
or race. Secondary associations occur across groups, so that one's 
Job associations or other formal relations extend beyond one*s ethnic 
group. 



Primary relations are those in which we are placed in intimate, face- 
to-face, personal, and Informal relations with others. They usually 
Involve the whole personality rather than the fulfillment of func- 
tions... 



Secondary associations , by contrast, are those in which our re- 
lations are casual, frequently functional, and usually not face- 
to-face. They tend to be formal and do not involve the whole per- 
sonality. (Green, p. 12) 



o 
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Finally, Uith " structural assltnUatlon ," both primary and sec- 
ondary associations occur across all groups so that one might find 

Interfalth marriage, as well as extensive Intermingling on the job. 

« 

In other words, ethnic or other differences do not dlsappear-*they 
are simply considered to be Irrelevant to any particular concern • 
In some ways, the testing movement heralds a de-emphasls of "relevant" 
differences In favor of universally prescribed operational slml- 
larltlesi 



Pluralism as an Ideal or reality provides an Inadequate account 
of the social context, however. Mannheim writes that In every society 
there Is a generally accepted interpretation of reality. He refers 
to this as the "reality level." "It Is a sociological fact that pub- 
lic thinking unconsciously establishes such reality levels, and a 
society Is only Integrated 1£ Its members roughly agree on a cer- 
tain ontologlcal order. "^^ He rejects the adequacy of the market 
liberal's view of Individualism for the social demands of coexis- 
tence and Integration* The "reality" he proposes Is democracy. "Dem- 
ocratic society. . .develops a level below which no citizen should fall... 
Democracy essentially admits competing reality levels to the realm 
of discussion, and adjusts these reality levels through communication, 
living contacts, exchange of Ideas, development of common rituals." 
In other words, a democratic system allows for the coexistence of 
diverse beliefs and priorities while providing the consensual framework 



It appears at first glance that "structural assimilation" Is simply 
"Insular pluralism" writ large. Both models are characterized by 
the Integration of primary and secondary associations. "Structural 
assimilation" contains an ambiguity which contributes to this dif- 
ficulty of distinguishing between the two models. On the one hand, 
"structural assimilation" does not Imply the necejsslty of any 
shared meanings or Ideology. No notion of "comiminlty" Is associated 
with this model. On the other hand, "structural assimilation" 
Implies a "homogeneity of v&levant characteristics." The home- 
genelty of relevance Implies consensus regarding the criteria de- 
termining what these characteristics might be — a consensus which Is 
similar to Mannheim's "reality level," or a sense of "community •" 
"insular pluralism" Is characterized by both a sense of community, 
and the occurrence of primary and secondary associations within the 
one group. ("Structural assimilation" Is characterized by asso- 
ciations occxxxting aowss groups.) 
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within which any conflicts nay be resolved. The resolution Mannheim 
offers Is a reaffirmation of a basic American tenet* 



The perils tent > countervailing thrusts toward homogeneity and 
diversity constitute a policy-relevant concern* They are policy-re- 
levant both In their effects on the process by which decisions are 
made and their effects on the substance of the decisions made* The 
tension may be characterized as being between diversity as Ideal or 
reality In a time of flux and change » and the Integrative consensus 
or homogeneity necessary for society. This issue underlies the po- " 
•lltlcal sensitivity of attempts to broaden the operational meaning of 
^^minimal skills'' in schools. 



Two Conceptions of the ''Minimal Skills" Necessary For Effective 
Functioning in Society? Dewey and Frelre 

If men are unable to perceive critically the themes 
of their time^ and thus to Intervene actively in 
reality » they are carried along in the wake of change. 
They see that the times ar. changing > but they are 
submerged in that change and so cannot discern its 
dramatic significance. And a society beginning to 
move from one epoch to another requires the develop*- 
ment of an especially flexiblei critical spirit. 
Lacking such a spirit » men cannot perceive the marked 
contradictions which occur in society as emerging 
values In search of affirmation and fulfillment clash 
with earlier values seeking self-preservation... 
Contradictions increase between the ways of being, 
understanding, behaving, and valuing which belong to 
yesterday and other ways of perceiving and valuing 
which announce the future. •. In such a phase man 
needs more than ever to be Integrated with his 
reality . [Emphasis added] 

(Frelre, Education for Critical Consciousness, pp. 7-8.) 



The themes of social change, flux and diversity are common enough. 
The pervasiveness of these ideas would seem to warrant speculation 
as to both the role of schooling in such a society and the skills 
which are necessary for effective .unctionlng within it. Frelre i 
for Instance, has some clear notions of "minimal skills." He writes 
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about the capacity for " cultural action or the ability to engage In 
evaJ.uatlve social activity} he writes about " praxis >" by which he means 
the Integration of action and reflection. "Minimal skills" here are 
predicated on a well articulated notion of "education" rather than 
"schooling." Frelre^s idea of the "educated man" is one who acts on 
and In his environment i who engages In the process of developing his 
world. The outcomes sought are ongoing and dynamic* Their assess- 
ment! needless to say> Is beyond the purview of existing standarlzed 
tests. 

If one puts aside the specifically revolutionary Intent of Freire*s 
work and relates it to the writing of Dewey > for Instance, common- 
alities in pedagogical procedure and purpose emerge. Recommendations 
for "problem-posing" education or "problematizing" and "authentic 
dialogue" between teacher and learner spaxi both works. Dewey also 
casts his educational philosophy and recommendations in the context 
of a political or social ideal—democracy — and seeks pedagogical meth- 
ods compatible with that ideal. For Inst ance. he writes that "the 
educational process is one of continual reorganization ^ of recon- 
structing and transforming"; of the necessity for "an actual empirical 

situation as the initiating phase of thought"; that democracy is "a 

19 

mode of associated living, of conjoint communicated experience." 
His writing contains the same rhetoric of self-determination and 
active participation as does Freire's. 

One critical difference between Dewey and Fr^ire, however, is 
that the former operationally constrains his prescriptions for educa- 
tion to "schooling." Schools have traditionally been conceived as 
institutions whose function it is, among other things, to transmit 
the "dominant" beliefs of the society they serve. Dewey lists cul- 
tural transmission as one of the primary responsibilities of schools. 
Freire, on the other hand, alerts one to the potentially oppressive 
nature of acculturation as a non-dialectical process. The difference 
between the two approaches is significant. Both men are concerned 
with "personal efficacy"; both men conceive of "personal efficacy" 
as occurring within, and being actively directed toward the social 
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environment* Preire^ however » stops short of describing the *'reality. 
level/* or the society which results from or provides the conditions 
for his idea of ^'education." That is, what mi^t or ought this con-* 
sensual environment look like? Dewey directly addresses this problem 
by outlining his conception of ^*democracy«^* He pursues the peda- 
gogical Implications of this notion in his recommendations that the 
schools replicate the democratic processes. The ^'minimal skills" for 
Dewey are these democratic, "action" skills. To the extent that Dewey 
specifies the sort of society for which these skills are intended, 
and requires replication of some aspects of this society by the schools, 
to that extent he is concerned with cultural transmission* 

One can relate the notions of "minimal skills" sketched above to 
Greenes notions of "half-way pluralism" or "insular pluralism." The 
consequences of either notion of pluralism for the substance and form 
of education may be characterized as follows. One finds on the one 
hand that "personal efficacy" might be based on a concern for adapt- 
ability — which implies not integratioh^ but conformity* In this in- 
stance one can talk, for example, about adopting certain white middle- 
class role skills ("minimal skills") in order to succeed in secondary 
relations, without relinquishing other ^ non-white non-middle-class 
core values for primary relations* This is consistent with the notion 
of "half-way pluralism." On the other hand, there is the concern for 
an identity of value, action, and intent, which implies not con- 
formity with partial commitment, but a holistic integration — Freire's 
"praxis" and "cultural action." This second interpretation of "per- 
sonal efficacy" may imply an "insular pluralism" as its ideal.* That 
is, the "personal efficacy" sense of^'minimal skills" here rests 
in part on an Integration of the primary and secondary relations. The 
environment for which the skills are intended becomes a "community" 
of shared and changing meanings which enables the "whole man" to be 
involved in both types of relations. The practical and ethical 



This is not to say that Freire would support "insular pluralism" 
as an ideal. Clearly, he would not. The Ideal of holistic in- 
tegration, however, may be said to be consistent with the notion 
of "insular pluralism." 
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problems then become respectively the reallzablllty and deelrablllty 
of such a ••community*' in a society such as 20th century United States* 

The Schools and "Minimal Skills for Personal Efficacy* * 

The issue of the school's responsibility for '^minimal skills for 
personal efficacy'* requires that the politically sensitive Issues 
raised in the preceding sections be translated into public policy. 
Such a translation depends in part on the viability or operational re- 
definition of community in our times. Further, It rests on the ex- 
tent to which It is considered appropriate or desirable that public 
policy intervene in those concerns which have been traditionally left 
to private determination-- that is » whether public policy may Inter- 
vene In the processes and outcomes of primary associations— for ex- 
ample > the family. The public-private dichotomy provides the frame- 
work for the discussion which follows. 

Essentially I the problem may be said to be the following; under 
what conditions Is the individual (or the individual together with 
his **primary assoqiates**) the proper source for determining the na- 
ture of his political or social obligation; or when Is the State the 
proper source? More important, perhaps, is the question as to under 
what conditions, and for which particular decisions > are both the 
State and the individual the proper sources for such determination. 
The issue is a recurring philosophical problem. It needs to be re- 
addressed In each era^ however; and ours Is no exception. To follow 
through on the discussion thus far, if one accepts Mannheim's notion 
of the necessity of a consensual framework, and Identifies it as 
''democracy," then an ideal and practice to which all citizens must be 
committed has effectively been Identified. It is "necessary" for 
society's functioning in the same way as is the labor force, for in- 
stance. Just as one asks (and the schools have in practice answered) 
the question: Is It the responsibility of the school to foster the 
"minimal skills" (however defined), for participation In the labor 
force, one must ask; is it the school's responsibility to foster* the 
^'minimal skills'' necessary for* the continuation of a democmtic society? 
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In other words > democracy has been proposed as a social i public ideal 
and necessity. Does the responsibility for teaching the skills for 
this public ideal and practice belong to the "public" institution for 
education — the school? This issue is critical and fundamental to "mini- 
mal skills" viewed from a social perspective. 

If one applies, admittedly rather crudely i these conceptions of 
"minimal skills" and "pluralism" to the social and political realities 
of the United States today, one finds evidence of the dilemmas out- 
lined being played out on the educational stage. One manifestation 
is a tension between the traditional "public" state and federal re-- 
sponsibility for education and general welfare, and the pressure for 
"private" determination of school procedure and of content, which 
might be translated as parent-teacher collaboration on issues of 
schooling. 

The current debate about the realm of "schooling" and the "mini- 
mal skills" encompasses an indecision about which notion of plurality 
Is to be espoused as the ideal — namely "half-way pluralism" or "struc- 
tural assimilation." Reference to both Ideals can be discerned in 
the debates about the control of schools. 

Some proponents of local control base their arguments on as- 
sumptions pertaining to the desirability or the moral necessity of 
maintaining certain group differences. Some notion of "half-way 
pluralism" could be seen to constitute the framework for such a view. 
Namely, if schooling is viewed as an institutionalized, formal ex- 
tension of the primary group (for example, the family), and its as- 
cribed function is to maintain and foster the priorities which are 



The complexities of the notion of democracy as they relate to the 
"public-private" distinction have Intentionally been omitted here. 
Operational and theoretical issues pertaining to the integration 
of individual or group rights with general welfare, for Instance, 
rare directly relevant to the question of public policy for edu- 
cation. The aim of this paper is to raise some of the Issues. 
Detailed analysis of these is subject matter for a further paper. 
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conalGtenc with the milieu In which it is located, local control of 
schooling would seem necessary* 

The proponents of centralized decision-making argue for an equal- 
ity of educational opportunity which disregards different Interpre- 
tations of achievement and emphasizes uniform student outcomes. This 
view is contained in the legislation for statewide testing. It is 
consistent with the lack of concern for differences which character- 
izes "structural assimilation." The locus of educational decision- 
making is ideologically irrelevant, as is the fostering of alterna- 
tives to schooling. A question then remains for policy-makers. Under 
which conditions do these ideals need to be reconciled? 

In other words, on what grounds can one make a case for federal 
and state responsibility for education? One has the historical, 
pragmatic reality—the fact of a publicly funded, institutionalized 
system whose purpose it is to teach those between the ages of 5 through 
18 years. This system includes schools, local, state and federal 
education agencies as well as various professional associations. 
Some of the agencies have ptriraarily coordinating and policy-making 
functions i others primarily the teaching function. One also has the 
strongly democratic principles which constitute the ideology on which 
the United States is founded, with one of the corollaries being the 
control of public education by the public. Of course, the agencies 
above are said to serve as representatives of the public interest. 
The problem in practice derives from the disjunction between the 
principles contained by the ideology, and the possibilities for action 
provided by the existing administrative structures. One of the conse- 
quences of bureaucratization is the effective neutralization of ex- 
ternally initiated acts whose realization depends on progressive 
movement through the organization's decision-making channels. In 
a situation such as thiS| "public'^ control of education takes on a 
rather narrowly circumscribed meaning. 




It is here that the problem of the legitimate parameters of "re- 
sponsibility" may need to be explicated and examined in relation to 
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the criteria chosen to define the "adequacy'* of schools. Are schools 
to be deemed adequate If they teach "social" as well as "cognitive" 
skills? In other words, should one of the criteria for "adequacy" be 
attention by the schools to the "democratic values" ? Local deter- 
mination of the objectives of schooling raises the possibility of al- 
ternative conceptions of skills necessary for effective functioning 
In varying conceptions oi society > with the accompanying claim that 
it is the school's responsibility to teach these skills. 

The criteria for determining the adequacy and functions of schools 
may change or vary considerably, depending on the administrative 
level on which these formulations of criteria are made. (A move- 
ment such as uniform, statewide testing, for Instance, would be seen 
as one thrust which makes such diversity difficult.) One might ask, 
for example, whether the skills specified by state agencies as nec- 
essary for effective functioning are compatible with the milieu which 
provides the "real" setting for the school and its students. There 
may be other skills, which are seen as equally crucial for "personal 
efficacy" for which the community demands the school be responsible. 
One example for a "minimal skill" other than the 3R's might be self- 
defense skills, another, consumer skills; or to use "skill" more 
loosely — knowledge about drugs. In defining "minimal skills" as the 
3R's, the danger is the sanctification of one conception at the ex- 
pense of others which may have equal legitimacy as the primary re- 
sponsibility of the school. The further issue which should then be 
addressed is whether the skills taught, the objectives of the school, 
ought in all cases be compatible with the demands of those who attend 
it. The question being raised is whos<^ goals should be taken as the 
goals of education. Under what conditions can pluralism be encouraged 
and strengthened, and under what conditions would such diversity so 
fragment the larger society that anarchy rather than functional syn- 
thesis would result? As indicated earlier in the discussion of 
the necessity of a "reality level" beyond individual priorities or 
desires, the school in some instances might have a responsibility 
to contravene individual or local expectations — and the state to 
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contravene local school expectations. 

Accountability legislation articulates a concern about the "ade- 
quacy and efficiency'' of schools. This concern might include some 
provision for addressing the range of conceptions and criteria avail- 
able for surh an evaluation. 



The Assessment Issue 

What would it mean then to implement an educational system which 
was consistent with the "personal efficacy" conception of minimal 
skills ) given the realities of the constraints imposed by the current 
pressure for an accounting of outcomes and the nature of American 
society? How would one talk about outcomes in a system which deals 
primarily with process? It would seem that the starting point is with 
what one recognizes as legitimately constituting assessment. Assess- 
ment, as operationalized in the recently enacted legislation, pri- 
marily utilizes the standardized test as its tool. In some states 
there is concurrent collection of demographic data, but the use of 
this data is as a correlate of the results on the essential test cri- 
terion. The evaluation is insttwnent-hoxxnd. It is bound in yet 
another way. Assessment, or the demand for information about the 
outcomes of schooling for accountability is tied to a particular 
time — that time in which the student is at school. The assumptions on 
which this constraint rests are problematic, at the very least. For 



To return to the legislation, it is clear that little debate would 
occur about the necessity of skills such as reading, writing and 
arithmetic. However, there nay be debate about hou) these skills 
are taught and how one defines what they are. The subject-matter 
or examples used may be viewed as misrepresentations of tho range 
of experiences available to the students. Where compulsory testing 
extends to other content areas, choice of perspective and em- 
phaiiis becomes more problematic. American history, for example, 
could be rewritten to show that revolution and insurrection is 
necessary for progress and the attainment of civil rights. This is 
an interpretation rarely represented in the history books or texts, 
and one can easily guess why. Furthermore, traditional conceptions 
of the contributions and history of various racial and ethnic groups 
in the United States are currently being challenged. 
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exaisplei is education, no less learning > being reduced to jjean the 
retention of skills between the ages of 6 and 18 years? It would 
seem that the outcomes of schooling relate to activities and skills 
which are made manifest for a reasonable length of time beyond the 
completion of 12th grade. Leaving aside for the moment the extremely 
complex problem of explicating the criteria by which one might Iden- 
tify these activities, and of establishing correlational links between 
particular school activities and later behavior, an alternative notion 
of assessment suggests itself. 

\n\at if assessment were understood to be a longitudinal process 
which allowed for collection of data over time — fov example over a 
period of ten years after completion of schooling? 

What if an "action"-based conception of outcomes were espoused — 
such as the one explored by Freire and Dewey — and criteria for selection 
were formulated? 

What if samples of the population were requested to periodically 
respond to either questionnaires which related to the outcomes cited 
above, or to participate in interviews, in order to provide some descrip- 
tive data? 
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The contradiction of using tests for the purpose of evaluating the 
sffectfc of schooling after its completion become apparent in pro- 
grams with even such carefully circumscribed parameters as the 
National Assessment of Educational Progress. NAEP uses the census 
mo^tel to test young adults, aged 26 to 35 years for retention of 
facts which should have been learned at school. There is an attempt 
to include an "action" component via for example, a self-report 
of "civic activities" or science Interests. If one applies the 
second notion of "minimal skills" as "necessary for personal 
efficacy" to this program> one could make the argument (which stands 
apart from other issues relating to the retention model which 
underlies the NAEP) that if those skills or knowledge which are 
being tested for are not retained or applied, they are in fact not 
necessary, and testing for them is to test for irrelevant con- 
cerns. In other words, rather than the knowledge or objectives 
being the standards or criteria for citizenship, writing or science, 
the nature of the responses become indicators of the necessity of 
these objectives for survival. 
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This last speculation raises a very sensitive issue — the dilemma 
of the privacy to which each individual is entitled, on the one hand; 
on the other, the concern for assessing the extent to which the goal 
of providing for each student the potentiality for self-determination 
has been met. Forced to its limits, the last speculation takes on the 
aura of an inquisition. This raises a central problem contained- in 
this conception of "minimal skills." Because one is dealing w^th moral 
and value questions — which are the proper domain of "education'^ — 
the problems of assessment force consideration of the distinction 
between "education" and the outcomes of "schooling," appropriate meth- 
ods of dealing with these, and most importantly, the propriety of 
assessing them. 

A further methodological issue must be addressed. The effort to 
quantify data is incompatible with the notion of "minimal skills" 
for personal efficacy. The survey technique described above is de- 
signed to provide quantified data. As such, It does not differ 
significantly from the standardized tests whojje utility I have dis- 
counted. It differs only in the rigor with which the information is 
collected and scored. This sense of "minimal skills," with its con- 
cern for self-actualization, requires that the individual's o^m concep- 
tion of the past, present and future be taken into account. The 
skills being assessed cannot be satisfactorily defined a priori . 
They cannot be described by stipulative or strictly aontingent de- 
finitions. If they could be, the dcientistic ethos contained in 
tests might be entirely appropriate. The assessment issue there- 
fore remains unsatisfactorily resolved. 

What is needed is a technique which combines "observer" or in- 
vestigator categories with "inner" or "subject -generated" categories, 



It seems that the incompatibility of the skills being sought with 
the quantif iabi J Ity of the data provided by surveys or tests rests 
on a paradigmatic (in Kuhn"s sense) difference which can be il- 
lustrated by the rules of evidence which apply to the biographer 
and those which apply to the scientist. The biographical historical 
mode is appropriate as an answer to the question of Identity to 
which this sense of "minimal skills" applies. 
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for when one talks about ''personal efficacy'' and "survivar' one is 

making strong allusions to self- deflnition. One such mode of analysis 

whose conceptualization, philosophical assumptions and methodology 

are consistent with the social conceptions of "minimal skills" is 

20 

participant observation. It provides the means for studying human 

meanings and social action as they are revealed in the context of 

society. It requires of the participant obssrver to see the goals and 

interests of people in the same way that the people see them, to see 

people in the concrete reality in which they present themselves in 

daily experiences; and to sense that people act freely within the 

scope of what they see as the possible, not as determined agents of 

0 1 

social forces as the traditional empiricists would see them. It 
is evident that this technique is based on the same assumptions of 
self-determination and action as the skills which it might assess or 
describe — skills which do not lend themselves to meaningful quantifi- 
cation. Participant observation has its roots in Max Weber* s formu- 
lation of the methodology of "verstehen. " It represents a synthesis 
of anthropological, sociological and literary techniques which have 
been widely utilized in studies of social phenomena. 

It is clearly beyond the scope of this paper to explore In any 
detail the methodological problems, feasibility and consequences of 
utilizing such a technique. It is simply suggested as one possibility 
whose viability might be discussed. One could certainly imagine 
community or district studies being undertaken. Prior to these, though, 
there would be a need to resolve a methodological issue pertaining to 
the grounds on which one would or could relate the manifestation of 
the skill one was studying to the process of "schooling." If it were 
decided, for instance, that the "personal efficacy" skills were the 
responsibility of the school, perhaps it would be necessary to have 
participant observers in the schools also, in order to obtain some 
information about the pvooessea of schooling which lend themselves to 
action and self-determination by students. One could imagine political 
pressures for and against such a procedure of evaluation. Moreover, 
the questions of evidence and criteria for evidence would have to be 
explored. How would one know whether someone is "personally efficacious"? 
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Would self-description or evaluation fluffice? Would observer criteria 
need to be introduced? The question relates both to theoretical or 
conceptual concerns, as well as to ones of methodology. And finally, 
the issue of the right to privacy would need to be addressed in rela- 
tion to the ideal of protecting each individual's right to an educa- 
tion, as well as the concern to assess that education's outcomes* 
This last issue is perhaps the most critical and difficult of all. 



The Technicist Ethos 

As noted above, the school might be viewed an an institutionalized 
transmitter of many of the myths for modern society. These myths per- 
tain to the existence of institutionalized, and therefore non-controll- 
able values, the quantif lability of attitudes and cognition, the 
fragmentability of knowledge, as exemplified in the school curriculum 

and tests, and finally, the myth of expertise — that one person's 

22 

judgment may determine what and when another person must learn. 
These notions are con3istent with, and serve to legitimate, a focus 
on means and techniques at the expense of explicit consideration of 
ends or goals. The preoccupation with techniques is the defining 
characteristic of the technicist ethos. Some of the recent account- 
ability legislation operatJLonalizes the "adequacy of schools'* in terms 
of a technique — the standardized test (or management techniques) 

rather than in terms of the possible or desirable functions of 

23 

schooling in our society. The problem is that techniques implicitly 
preclude consideration of a wide range of alternative goals. The 
"survival" sense of minimal skills explicitly addresses the function 
of education in a given social context. Articulation of goals is part 
of this formulation of minimal skills and, in a sense, determines how 
one defines them. Mandated tests for the 3R's resolve questions per- 
taining to the functions of schooling without addressing them directly. 

The legislation on statewide testing and accountability can be 
divided into two patterns. One is the state -mandated uniform testing 
and uniform conceptions of minimal skills; the other is the permissive 

See Appendix E (Table 3.1): Legislation for Educational Management 
Methods. 
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legislation for local determination of objectives. The second pat- 
tern does require confrontation with the technicist mode because it 
involves > at least in principle, consideration of goals* 



PART FIVE 
OONCLUSION 



The use of standardized tests represents an attempt to provide 
systematic data on the outcomes of schooling. The school's respon- 
sibility, as Indicated by the recent legislation, has shifted to the 
learning outcomes of students t The concern Is to provide a basis for 
evaluating the extent to which schools are executing their respon- 
sibilities. This emphasis on outcomes Is In part attributable to a 
search for educational equality. It may also, from the state govern- 
ment's perspective, provide a workable focus for a large, rather cum- 
bersome educational system with many "Inputs" and many "clients." 
Techniques for mass measurement abound i and In an age of computerized 
data collection, the task of comparing such results become^ easier. 
The complexity of the educational system i the accessibility of tech- 
nical facilities, and the mounting pressures for accountability all 
may have contributed to educational outcomes being operatlonallzed as 
test results on uniformly conceived and prescribed minimal skills. 
The attention currently being directed toward the Issue of account- 
ability by the lay public, professional educators and state legislatures, 
together with the moves to Incorporate the means by which this account- 
ability might be assessed In legislation, makes opportune the examina- 
tion of Issues which have a bearing on th<i sorts of decisions made. 
These Issues have to do with the way accountability Is defined, the 
domain to which It Is said to apply, and the means by which It Is 
assessed. The preceding analysis addressed some of the Implications 
of Introducing Into accountability legislation requirements for the 
teaching of basic skills whose nature and evaluation Is specified by 
that legislation. 

Clearly, test results provide Inadequate Information about the 
success or failure of schools. The problems of test measurement of 
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achievement and aptitude, and the limited utility of test results for 

evaluation of the efficacy of schools have been frequently brought to 

public attention. The recent legislation seems to ignore the contro- 

25 

versies generated by abuses of other test scores in the past. De- 
spite equally strong warnings from testing professionals about the 
constraints governing the correct use of standardized tests, state- 
wide testing legislation for accountability has been passed, heralding 

26 

another cycl« of potential test abuse. 

Moreover, if this legislation acts as a model for schooling 
might be, the model, as it now stands, needs careful evaluation, as 
the primary teaching function of schools has in effect been defined • 
Henry David, executive secretary. Behavioral Sciences Division, 
National Academy of Science, National Research Council, was quoted in 
the March 1973 NAEP Bulletin as cautioning against the "normative con- 
siderations and judgments'* involved in assessment. He said that these 
occur in the specifications of objectives and decisions about ac- 
ceptability of responses. "We are likely to become the victims of 
over-processed data,'' said David. '*One likely consequence is to de- 
rive an image of what the schools should do, are doing, could do 
better, should not be undertaking." Such an image would represent 
"powerful restraints" on our own imagination in dealing with educa- 
tion. Some of the legislation might indicate that the traditional 
expectations of schools have indeed not been reexamined in terms of 
the insights provided by learning theory and a social perspective. 

The current legislation locks into place accountability mechanisms 
which are unclear both as to which of the two notions of "minimal 
skills" are being addressed, and the consequences of intervention 
regarding either notion. The consequences of intervention relate to 
the legitimate locus of authority and control of educational decision- 
making. Uniform statewide testing indicates a move to centralized, 
state government control of educational planning. But there are 
other, socio-political shifts toward oormunity control of education 
which may call for a reconsideration of statewide prescription of 
skills and their evaluation. The legislation also affects the 
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substantive issue of the educational goals to which attention might 
be directed. The most devastating consequence with respect to goals 
of using test scores as measures of educational outcomes is the 
psychometric trivialization of these outcomes. Despite the articu- 
lated concern for quality of outcomes, the use of testing as a tool 
for the evaluation of quality might be seen to represent an actual 
concern with narrowly conceived quantifiable outcomes. 
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APPCNOJX A 

TABLC 1.1 THE ROLE Of F£0€ftAL fUNOINO IN TH6 EXPANSION 
OF STATE ASSE^ENT PROGRAMS 



STATE 



INITIAL FUND;N0 



CONTINUATION 
FUNDING 



i>ARTI€IPATION 



ALABAMA 
tmodd tot hcti 



Tittt i tod M 



SEA 



Urtiv«ftity of Alabama 



Af)KAN^S 
Nm^ Ai$«ssm#rti 



T da V, 502 



Titla III 
($40,000) 



Titfa Ml Aify^tory Council. 
E PJC Oiv^riifiad Syitami 
CorpofitiOA of Tu<ton, 
n^on^ p«praian(ativ«> of 
partkipilirv^viiooli 



COLORADO 
E«atual»on Project 



Tiita lV,402aod 
SEA 



SEA (1972) 



Pilot pl«finih9 includad 
crou-tacti'^nal advisory 
(roufi liv^ Ie9<st«t0ft, 
eitlttAi, and profatsional 
•ducatOr$ 



DEt AWARE 
Edu€«1toiul 
Aceouniabtlity 
Syitcm (tyni^atim 
tyaiualton »M tts*stmaol f<x 
Tida I Migrant 
Piro^am. Tula III. 
and t^o Oatawart 
prograrm) 



Titla IV, 402 
<$d6.000 iot 3 
yaari b«ginn!i>4 
July M970I 



Advice from Educational 
Aceo-jntabitity Cooncil; 
icoriog ripOriir>gby 
NartOurt, Sract, J9vlr>0v{ch. 
1fk.:addit»oral v^alytM 
by Laht^ Unryanity 



Florida O^alopmtfltcwt*: O0«f«tjooal Fomli; P»Ofr»mlif#via«*dby 

Ptan for Educational Ftortda Edutattonal Stila ipproprlHiOnl vaHotiyoupi 

A<s«{$m«<it RaMarch *nd Devafopmtrtt tstpftttrmttd by 

Program Tida IV,402*rtAual 

graht* 



GEORGIA 
AtMiimafll 
P/Oi#ct 



Stata Mtd fedaral 

fundi 



I xpHi ma}or furtdirrg 
from itata lag^tation 



Son>« act>vit}at l»V«d 
(ohi^«r education 
(niiiiuti'oAi 



lOMA 
Gukdanca Surveys 



Stati and Titfa V 
rnonay 41970) 



Sutiiod Title Ml 
41971) 



KANSAS 
Edocabooal 
Informtttort System 
Oay«lopm«f»1 



ESEATidliV 
192,000 f lni ya«f 



Selact School P^actkai 
Efficlanry Committaa (autfiOft*«d 
by >tat« lag^datkKi) hki rKafvtd 
brianngi 



KENTUCKY 
Naadi Aucumant 
Stucfy,Ph«i*l 



1969 Titla in plan 
wai pracurior 



SEAfuTHk and 
Titla iV.40a 



NtOdf Ataatamant CrOup tnd 
EPICDtv#rtitiarfSyitam» 
Corporate of Tucton, ^jiioot 



LOUISANA 
Surety of Edoealional 



Titla III $S«.^60 
Tfdafl Sfd.60d 



Nof**»it»m Stita Unl^tnity »ftd 
Titfa Ml Advr»ory CouiKit 



frTc 
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^ 
M 



STATE 



TABLt l.t THE ROLE OF ^EDtRAL FUNDING IN THE EXPANSION 
OF STATE ASSESSMENT PROGRAMS 



INITIAL FUNDING 



CONTINUATION 
FUNDING 



PAftTIClPATJON 



MAINC 

Plan for "Onootn^ 
Ait«»mtot" 



Titit l.lll. IV 



UnMriity of Mtin* 



MONTANA 
Evilu4t>(y^ iot 

PtjATin^ and 
Daciiiofl -Making 



TnfilV.4W 



Advl»o<y board of f wr 
departmonts of public 
in$tn»ct>on itaff mambefi 



N€W HAMPSHIRE 
Stata Tatting 
Pro-am (global 
ingoali and 
pbjcctivat] 



(Propoi/d) Titii IV. 
402 primary; ESCA 
Tidall.lll.VI; 
itata and loc4l fundi 



Univirtity^f Ntvv Hwnpihira 



NCWJERSIY 
Our Schoof f 
Program 



Tit)allM1$d8>1de9) 



«70 - 1»7» Titfi IV, 
4O2{i^,0O0h Ntwiartay 
Bankirs ^locia(ion 



AdvliOTY Council on Edutalionai 
Naadi ^Mtsmant 



NEW YORK 
Pirformarica 
Indicators \n 
Education 
(Pilots 

Pupil Evaluation 
Proyimi (ilata-iiyida 
»choof-by>«ch6ol 
invintorv) 



TitlaV;SCAftaH 
(1970) I12O.00O 



TiiJal (IMS) for 
admfnbtration 



Cxpanikm pfM^nad; 
funding lourct not 
indwaiad 



SCAtndTilTtl 
($40,000) 



(In plan d«v«iopm«nt^ ETS; 
AST AaiociatM: RanfM^aar 
Ra««Kdf Corporation 



NORTH CAROLINA 
Proiact ad Itata -vvide 
au«timant(1971 ^ 1972) 



UniMdportioni of 
Tfda I^HI 
whara appropriata; 
S^A fundi (p*ndin| 
bill r*qu*»ti 1750.000) 



Propoi*d Education Davtfopmant 
Council (dtiztni): 
Oivliion of Community CoHagat ftnd 
Board of Hi^ar Education: outstda 
contfKtort fof ddta procais^nfl 



:WRTH DAKOTA 
Stata 'Wida Tatting 
Program 

Asmnnant of 
Educational Naadi 



Titlaliim.OOO) 
Tftia iV {$7,S60) 



Titfa m i2A%): 
participating 
Khooti \7B%) 



Planning Con>rhitte« and ad hoc 
tmt M^n naad ariiai 



Unlvtrtity of North DakoU 



OREGON 

CompfihaniJya 
program (tnhtittad but 
not impramtntadl 



Tida IV, 402 
Tid«V«&0« 



Not ipacf fiad 



Initituti for Educational 
Enginaarlng. coordinating 
^tf^ Atiliiant Suparintandant 
fof Plaining and frduttion 
and Data Procttting: Nonhwait 
RagiOfW Edueationil Laboratory 
<coniuttintr) 
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D AF>P£NOIX A (CONCLUOEO) 
I 

N TABLE 1,1 THE f^OLE OF FEDERAL FUNOIMO (\ THE EXPANSION 

<vl 0^ StATE ASSESSMENT (PROGRAMS 

O 

^ CONTINUATION 

O STATE JNITML FUNDING FUNDING PAftTtClPATION 



PENNSYLVANiA 
Educational 
Quality AiMiimant 



SEA(t967) 



Somt Title III 
fundi for data 
8r\«lY>ii«^d 



Board Committee oa Ou^ity 
Edueatk.0 «nd Stjte-wlde 
AdvisofV CoAimitiM 



SOUTH CAl^OLiNA 1968 TiiTe III Title iV. 402 Reaction paneli (local 

Five -Year Plan Needs A»ie«>»nent ichool perionrtel aM other 

to Improve edueatonl; cooiulianti 

Edueation 



TENNESSEE State fund*; Cor**ultants from Memphii 

Nee* Aiietiment Title III, I, IV, State Dnfvertlty 

Yeari 1,2.3 402 
(in planning 
itage) 



TEXAS 
Pilot PrOyim 

I, Pupil Appraiial: Cognitive Domairt 

IC Academic PerformarKe 

11 M^ow -Up Syi(em» / FoMow -Up Studies 

Ml. Pupil Appraisal; Aiiectrve Domain 

iV, School and Community Aiteumert Studies 

federal (Title 111). Contractors; regional 

state, and local ttalfs 

funds 



UTAH TitfelfiandIV «.000 c}ti»n*;33 educitk)nal 

Needs Assessment l$25,000) enperts; 7 selected organ liatloni 

f*roiect ki sUte 

State -wWeEvaTuation title /V. 402 (2/3) Eacft major proyam Workshop for reactioni pf 

System Title III {1/3) participating (n district end regional 

($4S.OOO) theiyttem personnel end evaluation 

ipeeiairsto 



VERMONT 
Design for Education 
Phase 1 of 4 
phases completed 
January 1. 1970 



State andTfdelil 



Task Force; 
piiaf of Elementary 
Educatkkn and five 
state elementary 
consultants (no 
standardised te<ts; 
loCalf irtictpat^on 
of ^.anb and citizens) 



WEST VIRGINIA 
Learner -drionted 
Assessment 
(proiected 
1971 - 1972) 



Table II (sic) 
(ma^H source): lome 
state funds 



Source; Cwhpiled from Stat* Educational Aiseninent Proyar^H, Prlntelbn. Naw Jeney: Educational Testing Sarvtc*. 1971. ThU wmmaiv include* tv^enty-fiyt sUtei 
repori;ng asseeiment programs broader tfian re<}uirenients for Title I or Title III proyami and not funded eolefy by Title i W Tide III mpnle*. 
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APPENDIX B 
TABLe2.2 

ACCOUNTABILITY IN EDUCATION: LGGlSLAflON STUDIED 



State 


Date 


Statute 


Alaska 


1970 


Chapter 35 


Ariiona 


1969 


Article 2.1, Reading Achievement Tests, 1969 




1972 


Senate Bill 1294 


VHII 1 lUi illd 




uaiiTornia ocnooi Testing Act 




l9t>0 


iviiiier'Unrun oasic Heading Act 




1Q7 1 
15>/ 1 


Assemoiy tsm 2vo, Cnaptef 3o1 




1Q79 


Assembly Bill 665 (Amends testing legislation) 




1967 






1971 


Assembly Bill 2800 




1969 


Assembly Bill 606, Chapter 784 




1970 


Assembly Bill 1923, Chapter 1023 




1971 


Assembly Bill 2999, Chapter 1220 




1971 


Assembly Bill 1483, Chapter 1600 


Colorado 


1971 


Educational Accountability Act 




1971 


Senate Bill No. 42 (Pf BES) 4 Article 42 


Connactlcut 


1971 


Public Act No, 665 




1971 


Public Act No. 383 




1971 


Public Act No, S2 




1971 


Pgblic ActNo, 326 






rUPIIC MCI viQ, Oo£ 




1971 


Public Act No, 430 




1972 


Substitute Hou$e BUI No. 6371 




1972 


Public Act No, 204 


Florida 


1970 


Chapter 70-399 




1971 


Chapter 229,67 (Accountability Act) 




1969 


Chapter 231.29 


Hawaii 


1970 


Act 186 




1972 


Senate Resolution No, 190 




1972 


House Concurrerit Resolution No. 43 


fllmols 


(197^1 


{Senate Bill No, l43Q-did hot pass] 




(1972] 


(Senate Bill No, 1432*^id not pass] 




1972 


Senate Bill No. 1548 


Indiana 


1971 


Public Law No. 309 


Maina 


1967 


Resolution of the Senate 


Maryland 


1972 


Senate Bill No. 166 


MaiidCfiusatts 


1965 


Willis Harrington Act 


Miohioan 


1970 


Enrolled House Bill No. 3886 


Nebraska 


1969 


Legislative Bill No. 959 



TABLE 2.2 continued 



StMe 


Date 


Statute 


New Jersey 


1972 
[1972] 
1971 


Governor's Message 
(Assembly 8ill No. S22-'did not pass] 
Senate Bill No 2233 


New Mexico 


1967 
1971 


Chapter 16,s,59 

VWIdpioi MOM 

i>enate Memorial No. 40 


Ohio 


1972 


House Bill No. 475 


Oregon 


1971 


Senate Bill No. 131 


Pennsylvania 


1963 


School District Reorganization Act 


Rhode Island 


1969 
1963 


Chapter 49, s. 16 
Chapter 16-22 


South Dakota 


1969-71 


Chapter 62, Session Laws of 1969 (amended in 
1970 and 1971) 


Virginia 


1971 
1972 


s. 2, Article VIM of Constitution 
H.S45 


Washington 


1969 


28A, 67.065 


Wisconsin 


1971 


s.443,Ch.225 



States where accountability legislation was introduced in 1972 but did not pass (6) 



Ataska* Minnesota* * - Text of Wfl not 

Idaho* New Jersey available for 

Illinois (2 of 3 bills) Oklahoma* examination 



States where dccountabitity legislation may be Introduced In 1973 (16) 



Connecticut Massachusetts Oklahoma 

Georgia Minnesota Oregon 

IMinots Nevada Rhode Island 

Indiana New York Texas 

Kansas North Carolina Wyoming 
Maryland 



Source: Statutei, billi, and ret«t«d infotmMion reproduced by iht Cooperative Aeoduntabilrty i^oiect. May and 
October 1&^2, and ApfiMd73. 
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APPENDIX C 
TABLE 2.3 

STATUS OF ACCOUNTABILITY LEGISLATION, FALL 1972 



Stdt6 


Legislation 


Legislation may be 


None enacted as 




cnacica 


intrAriiiivarl in iQ*7^ 
iiiiiuuui^a Ml liifo 


Af Call iQlO 

01 rail, 


Alab3rt>a 






X 


Alaska 


X 






Arizona 


X 






Arkansas 






X 


Ca1iforrii9 


X 






Colorado 


X 






Connecticut 


X 


X 




Delaware 






X 


District of ColumL'a 






X 


Florida 


X 






Ceorgia 




X 


X 


Hawaii 


X 






Idaho 






X 


Illinois 


X 


X 




Indiana 


X 


X 




Iowa 






X 


Kansas 




X 


X 


Kentucky 






X 


Louisiana 






X 


Maine 






X 


Maryland 


X 


X 




Massachusetts 


X 


X 




Michi^n 


X 






tfl II II io>u \o 




V 


' V 
A 


Mississlnr^ 






X 


Missouri 






X 


Montana 






X 


Nebraska 


X 






Nevada 




X 


X 


New Hamoshire 






X 


New Jersey 


X 






NeW Mexico 


X 






New York 




X 


X 


North Carolina 




X 




North Dakota 






X 


Ohio 


X 






Oklahonia 




X 


X 


Oregon 


X 


X 




Pennsylvania 


X 






Rhode Island 


■■ X 


X 




South Cdmlina 






X 


South Dakota 


X 






Tennesee 






X 


Texas 




X 


X 


Utah 






X 


Vermont 






X 


Virginia 


X 






Washington 


X 






West Virginia 






X 


Wisconsin 


X 






Wyoming 




X 


X 


TOTAL 


23 


16 


2d 



Soort*: Cooperaiiv* AocountaWity f>roi*cl t\^t, Ajyil 1973, vf. 



APPENDIX D 



Summary of Statewide Testing Programs 

prepared by Educational Testing Service in collaboration with Educa- 
tion Commission of the States and Education Resources Information 
Center (Princeton, N.J.: Educational Testing Service, 1971), 



Many of the authorizations from legislatures are principally for the 
assessment of education by tests. That is, there is a mounting legislative 
pressure for documenting the products of the educational process by state- 
wide testing programs. Some states have already set in motion widely rang- 
ing programs of tests (Pennsylvania and Michigan being notable examples), 
and others report themselves to be at the point of doing so (among them 
Colorado and Delaware). Some states are starting with rather narrow content 
coverage, but are planning for massive programs later on (Florida and Georgia, 
for example) . 

The content of most current state testing programs—whether mandated or 
unmandated by legislative bodies--is often less surprising than it is signifi- 
cant • The states engaged in some form of assessment-by-testing are mainly 
concerned with how well their educational systems are succeeding in imparting 
basic skills. Only relatively fe w go beyond the 3 Rs . Arizona, for example, 
received a mandate for the Arizona State Third Grade Reading Achievement 
Program, to begin this year* Although the specif ic objectives of the program 
are not yet available, strong effort will apparently be made to provide back- 
ground data to lend depth and perspective in Interpreting test scores/ 

As another example, recent legislation In Michigan calls for measures 
of the basic skills at grades 4 and 7. This program, which Is now In Its 
second year, covers verbal analogies , reading, English (mechanics of written 
English), and mathematics* In the first year, only average scores by school 
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and school district were reported, since the tests were consciously designed 
to be short to yield adequately reliable scores on individual students. This 
approach^ however, was changed for the 1970-71 administrations* Tests are 
now of conventional length to provide the schools with information concerning 
the achievement of individuals. Although the major stress here has been on 
the academic areas, the Michigan program has also given some attention to 
assessing the influence of schooling on student aspirations* 

California, which has a history of mandated testing programs going back 
to 1961, is another instance where testing of the basic skills has been 
strongly emphasized* In 1965, the Miller-Unrah Basic Reading Act created an 
obligatory testing program in reading for the primary grades* This concentra- 
tion on the basic has been further reinforced by a recent legislative require- 
ment for the adoption of minimum academic standards for certain grades and 
the selection of tests to be used statewide in evaluating the attainment of 
these standards. 

Florida, for example, is presently concerned with measuring only 
achievement in reading, but is also planning a most ambitious program that 
will sample students in kindergarten through grade 12 In all the basic subjects* 
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p APPENDIX E 

N TABLE 3.1 



LEGISLATION FOR EDUCATIONAL MANAGEMENT METHODS 



State 


D3te< 


PPBS 


Alaska 


1970 


X 


Arizona 


1972 




California 


1967 


X 




1971 


X 


Colorado 


19Z1 


X 


Hawaii 


1970 


X 


Illinois 


1972 


X 


Indiana 


1971 


X 


Nebraska 


1969 




New Mexico 


1967 




Ohio 


1972 


X 


Rhode Island 


1969 





ll 



Source: Derived from examination of re^fslation reported to the Cooperative Accountability Project, 1972 sod April, t9?d 
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FtK>m Federal to State Stntotu^ee: The Bux*den of Aooomtdbilitu 

The search for a new definition of the federal-state partner-- 
ship in education marks a moment of change. Such time of transition 
provides a unique opportunity to review the past^ to examine the 
present^ and to explore alternative futures* 

Since entering the domain of elementary and secondary education^ 
the Office of Education has functioned with an unprecedented degree 
of '^accountability.^' Congressional mandates clearly define spheres 
of activity and the intent of ike legislation must he carefully 
ohsewed* Both executive and legislative branches demand quanti- 
fidble evidence of success i Evaluations of the results of activity 
are public information^ available not only to decision-^makers in the 
form of reports^ but to the general pvi^lic through the national press 
Current concern with accountability at the state level^^with defining 
areas of state responsibility and with measuring outcomeS'-'^indicate 
a wide currency of this mode of operating. As the Office of Eduoa^ 
tion has discovered^ such pressures for accountability have an 
enormous impact on the administering agency* 

At the present moment i accomtability is a word With multiple 
definitions* In its working form^ two broad patterns are clear* 
In one mode^ accountability is a new articulation of an old demand 
for efficiency in schools. It reinforces the tradition of the in- 
dustrial metaphor applied to education^ using input-output models 
which ignore human and social costs and benefits. In a second f^rms; 
it challenges the industrial metaphor by forcing an examination of . 
the goats of education and a claHfication of areas of responsi- 
bility* The movement is recent and its future is uncertain* It 
hotds^ however^ a two- fold potential: to infuse new lif$ ini^ 
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